Apache Pig GRUNT Shell Usage

posted on Nov 20th, 2016

Apache Pig

Apache Pig is a high-level platform for creating programs that run on Apache Hadoop. The language for this platform is called Pig Latin. Pig can execute its Hadoop jobs in MapReduce, Apache Tez, or Apache Spark. Pig Latin abstracts the programming from the Java MapReduce idiom into a notation which makes MapReduce programming high level, similar to that of SQL for RDBMSs. Pig Latin can be extended using User Defined Functions (UDFs) which the user can write in Java, Python, JavaScript, Ruby or Groovy and then call directly from the language.

Pre Requirements

1) A machine with Ubuntu 14.04 LTS operating system

2) Apache Hadoop 2.6.4 pre installed (How to install Hadoop on Ubuntu 14.04)

3) Apache Pig pre installed (How to install Pig on Ubuntu 14.04)

Pig GRUNT Shell Usage

The Grunt shell of Apache Pig is mainly used to write Pig Latin scripts. Prior to that, we can invoke any shell commands using sh and fs.

$ cd /usr/local/pig/bin

Step 1 - Change the directory to /usr/local/pig/bin

$ cd /usr/local/pig/bin

Step 2 - Enter into grunt shell in local mode.

$ ./pig -x local

OR

Step 3 - Enter into grunt shell in MapReduce mode.

$ ./pig -x mapreduce

Shell Command

Using sh command, we can invoke any shell commands from the Grunt shell.

sh ls 

fs Command(HDFS)

Using the fs command, we can invoke any FsShell commands from the Grunt shell.

fs -ls

Utility Commands

The Grunt shell provides a set of utility commands. These include utility commands such as clear, help, history, quit, and set; and commands such as exec, kill, and run to control Pig from the Grunt shell.

clear

help

history

set default_parallel=no_of_reducers
set debug=on/off
set job.name=job_name
set job.priority=very_low/low/normal/high/very_high
set stream.skippath=for streaming

quit

exec /home/hduser/Desktop/PIG/sample_script.pig

run /home/hduser/Desktop/PIG/sample_script.pig

dump;

Please share this blog post and follow me for latest updates on

facebook             google+             twitter             feedburner

Previous Post                                                                                          Next Post

Labels : Pig Installation   Pig Execution Mechanism   Pig Load and Store Operations   Pig Diagnostic Operators   Pig Group Example   Pig Join Example   Pig Cross Example   Pig Union Example   Pig Split Example   Pig Filter Example   Pig Distinct Example   Pig Foreach Example   Pig OrderBy Example   Limit Example   Pig Eval Functions Example   Pig BagToString Example   Pig Concat Example   Pig Tokenize Example   Pig UDF's Java Example   Pig SCRIPT