Apache Pig Tutorial – Executing Script with Parameters

Goal of this tutorial is to learn Apache Pig concepts in a fast pace. So don’t except lengthy posts. All posts will be short and sweet. Most posts will have (very short) “see it in action” video.

In the previous post, we saw how to run Pig Latin instructions as a script. But there is one problem with that; we hard coded the Input and the Output location and it is not ideal. In this lesson we will see how we can pass parameters to the script.

Parameter Placeholders

Take a look at the below sample. ‘$input’ is a parameter which defines the input location and ‘$output’ is a parameter which defines the output location.

prices = LOAD '$input' USING PigStorage(',') as (exchange:chararray, symbol:chararray, date:datetime, open:float, high:float, low:float, close:float,volume:int, adj_close:float);

STORE top10 INTO '$output' USING PigStorage(',');

Passing Parameters

Now we know how to set the parameters, let’s see how to pass parameters from the command line. Use -param  and the parameter name to set and pass the value for the paramaters. In the below sample, at run time the value of the $input  in the script will be substituted as /user/hirw/input/stocks

pig -param input=/user/hirw/input/stocks -param output=output/pig/avg-volume-params /hirw-workshop/pig/scripts/average-volume-parameters.pig

 See It In Action

Previous Lesson : Running as a Script

 Next Lesson : Tuple & Bag