[cloudera@quickstart ~]$ pig -x local citywise_avg_students_age.

pig
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
2021-11-14 22:30:58,779 [main] INFO org.apache.pig.Main - Apache Pig version
0.12.0-cdh5.12.0 (rexported) compiled Jun 29 2017, 04:34:31
2021-11-14 22:30:58,779 [main] INFO org.apache.pig.Main - Logging error messages
to: /home/cloudera/pig_1636957858756.log
2021-11-14 22:30:58,796 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - user.name is deprecated.
Instead, use mapreduce.job.user.name
2021-11-14 22:30:59,772 [main] INFO org.apache.pig.impl.util.Utils - Default
bootup file /home/cloudera/.pigbootup not found
2021-11-14 22:30:59,955 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated.
Instead, use fs.defaultFS
2021-11-14 22:30:59,959 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
2021-11-14 22:30:59,966 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to
hadoop file system at: file:///
2021-11-14 22:31:00,166 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated.
Instead, use fs.defaultFS
2021-11-14 22:31:00,175 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
2021-11-14 22:31:00,375 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated.
Instead, use fs.defaultFS
2021-11-14 22:31:00,383 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
2021-11-14 22:31:00,540 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated.
Instead, use fs.defaultFS
2021-11-14 22:31:00,545 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
2021-11-14 22:31:00,662 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated.
Instead, use fs.defaultFS
2021-11-14 22:31:00,665 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
2021-11-14 22:31:00,755 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated.
Instead, use fs.defaultFS
2021-11-14 22:31:00,756 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
2021-11-14 22:31:00,835 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated.
Instead, use fs.defaultFS
2021-11-14 22:31:00,837 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
2021-11-14 22:31:00,917 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated.
Instead, use fs.defaultFS
2021-11-14 22:31:00,921 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
2021-11-14 22:31:01,042 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated.
Instead, use fs.defaultFS
2021-11-14 22:31:01,043 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
2021-11-14 22:31:01,637 [main] INFO org.apache.pig.tools.pigstats.ScriptState -
Pig features used in the script: GROUP_BY
2021-11-14 22:31:01,701 [main] INFO
org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer -
{RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, DuplicateForEachColumnRewrite,
GroupByConstParallelSetter, ImplicitSplitInserter, LimitOptimizer,
LoadTypeCastInserter, MergeFilter, MergeForEach, NewPartitionFilterOptimizer,
PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter],
RULES_DISABLED=[FilterLogicExpressionSimplifier, PartitionFilterOptimizer]}
2021-11-14 22:31:01,748 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation -
mapred.textoutputformat.separator is deprecated. Instead, use
mapreduce.output.textoutputformat.separator
2021-11-14 22:31:01,880 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File
concatenation threshold: 100 optimistic? false
2021-11-14 22:31:01,898 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer -
Choosing to move algebraic foreach to combiner
2021-11-14 22:31:01,941 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer -
MR plan size before optimization: 1
2021-11-14 22:31:01,941 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer -
MR plan size after optimization: 1
2021-11-14 22:31:02,017 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - session.id is deprecated.
Instead, use dfs.metrics.session-id
2021-11-14 22:31:02,018 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Initializing JVM Metrics with processName=JobTracker, sessionId=
2021-11-14 22:31:02,057 [main] INFO org.apache.pig.tools.pigstats.ScriptState -
Pig script settings are added to the job
2021-11-14 22:31:02,186 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation -
mapred.job.reduce.markreset.buffer.percent is deprecated. Instead, use
mapreduce.reduce.markreset.buffer.percent
2021-11-14 22:31:02,186 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler -
mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2021-11-14 22:31:02,186 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.output.compress is
deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2021-11-14 22:31:02,188 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler -
Reduce phase detected, estimating # of required reducers.
2021-11-14 22:31:02,189 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler -
Using reducer estimator:
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstima
tor
2021-11-14 22:31:02,191 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstima
tor - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=323
2021-11-14 22:31:02,191 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler -
Setting Parallelism to 1
2021-11-14 22:31:02,191 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.reduce.tasks is
deprecated. Instead, use mapreduce.job.reduces
2021-11-14 22:31:02,242 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler -
Setting up single store job
2021-11-14 22:31:02,258 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Key
[pig.schematuple] is false, will not generate code.
2021-11-14 22:31:02,258 [main] INFO org.apache.pig.data.SchemaTupleFrontend -
Starting process to move generated code to distributed cache
2021-11-14 22:31:02,258 [main] INFO org.apache.pig.data.SchemaTupleFrontend -
Distributed cache not supported or needed in local mode. Setting key
[pig.schematuple.local.dir] with code temp directory: /tmp/1636957862258-0
2021-11-14 22:31:02,402 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1
map-reduce job(s) waiting for submission.
2021-11-14 22:31:02,404 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker.http.address
is deprecated. Instead, use mapreduce.jobtracker.http.address
2021-11-14 22:31:02,414 [JobControl] INFO org.apache.hadoop.metrics.jvm.JvmMetrics
- Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2021-11-14 22:31:02,761 [JobControl] WARN
org.apache.hadoop.mapreduce.JobResourceUploader - No job jar file set. User
classes may not be found. See Job or Job#setJar(String).
2021-11-14 22:31:02,828 [JobControl] INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to
process : 1
2021-11-14 22:31:02,828 [JobControl] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths
to process : 1
2021-11-14 22:31:02,846 [JobControl] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths
(combined) to process : 1
2021-11-14 22:31:02,900 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter
- number of splits:1
2021-11-14 22:31:02,908 [JobControl] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated.
Instead, use fs.defaultFS
2021-11-14 22:31:02,909 [JobControl] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
2021-11-14 22:31:03,162 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter
- Submitting tokens for job: job_local2048462731_0001
2021-11-14 22:31:03,554 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The
url to track the job: http://localhost:8080/
2021-11-14 22:31:03,555 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher -
HadoopJobId: job_local2048462731_0001
2021-11-14 22:31:03,555 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher -
Processing aliases citywise,citywise_avg_age,student_info
2021-11-14 22:31:03,555 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher -
detailed locations: M: student_info[1,15],student_info[-1,-
1],citywise_avg_age[5,19],citywise[3,11] C: citywise_avg_age[5,19],citywise[3,11]
R: citywise_avg_age[5,19]
2021-11-14 22:31:03,563 [Thread-7] INFO org.apache.hadoop.mapred.LocalJobRunner -
OutputCommitter set in config null
2021-11-14 22:31:03,616 [Thread-7] INFO
org.apache.hadoop.conf.Configuration.deprecation -
mapred.textoutputformat.separator is deprecated. Instead, use
mapreduce.output.textoutputformat.separator
2021-11-14 22:31:03,618 [Thread-7] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated.
Instead, use fs.defaultFS
2021-11-14 22:31:03,618 [Thread-7] INFO
org.apache.hadoop.conf.Configuration.deprecation -
mapred.job.reduce.markreset.buffer.percent is deprecated. Instead, use
mapreduce.reduce.markreset.buffer.percent
2021-11-14 22:31:03,618 [Thread-7] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.reduce.tasks is
deprecated. Instead, use mapreduce.job.reduces
2021-11-14 22:31:03,618 [Thread-7] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
2021-11-14 22:31:03,619 [Thread-7] INFO
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer
Algorithm version is 1
2021-11-14 22:31:03,619 [Thread-7] INFO
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - FileOutputCommitter
skip cleanup _temporary folders under output directory:false, ignore cleanup
failures: false
2021-11-14 22:31:03,622 [Thread-7] INFO org.apache.hadoop.mapred.LocalJobRunner -
OutputCommitter is
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter
2021-11-14 22:31:03,694 [Thread-7] INFO org.apache.hadoop.mapred.LocalJobRunner -
Waiting for map tasks
2021-11-14 22:31:03,697 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.LocalJobRunner - Starting task:
attempt_local2048462731_0001_m_000000_0
2021-11-14 22:31:03,831 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer
Algorithm version is 1
2021-11-14 22:31:03,835 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - FileOutputCommitter
skip cleanup _temporary folders under output directory:false, ignore cleanup
failures: false
2021-11-14 22:31:03,871 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ]
2021-11-14 22:31:03,879 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 323
Input split[0]:
Length = 323
Locations:

-----------------------

2021-11-14 22:31:03,897 [LocalJobRunner Map Task Executor #0] INFO

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader -
Current split being processed file:/home/cloudera/pig/sample_student_data.txt:0+323
2021-11-14 22:31:04,063 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.MapTask - (EQUATOR) 0 kvi 26214396(104857584)
2021-11-14 22:31:04,063 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100
2021-11-14 22:31:04,063 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.MapTask - soft limit at 83886080
2021-11-14 22:31:04,063 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600
2021-11-14 22:31:04,063 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.MapTask - kvstart = 26214396; length = 6553600
2021-11-14 22:31:04,105 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.MapTask - Map output collector class =
org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2021-11-14 22:31:04,163 [LocalJobRunner Map Task Executor #0] INFO
org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will
not generate code.
2021-11-14 22:31:04,209 [LocalJobRunner Map Task Executor #0] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Ma
p - Aliases being processed per job phase (AliasName[line,offset]): M:
student_info[1,15],student_info[-1,-1],citywise_avg_age[5,19],citywise[3,11] C:
citywise_avg_age[5,19],citywise[3,11] R: citywise_avg_age[5,19]
2021-11-14 22:31:04,230 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.LocalJobRunner -
2021-11-14 22:31:04,231 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.MapTask - Starting flush of map output
2021-11-14 22:31:04,231 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.MapTask - Spilling map output
2021-11-14 22:31:04,231 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.MapTask - bufstart = 0; bufend = 130; bufvoid = 104857600
2021-11-14 22:31:04,231 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.MapTask - kvstart = 26214396(104857584); kvend =
26214368(104857472); length = 29/6553600
2021-11-14 22:31:04,292 [LocalJobRunner Map Task Executor #0] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine -
Aliases being processed per job phase (AliasName[line,offset]): M:
student_info[1,15],student_info[-1,-1],citywise_avg_age[5,19],citywise[3,11] C:
citywise_avg_age[5,19],citywise[3,11] R: citywise_avg_age[5,19]
2021-11-14 22:31:04,301 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.MapTask - Finished spill 0
2021-11-14 22:31:04,304 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.Task - Task:attempt_local2048462731_0001_m_000000_0 is
done. And is in the process of committing
2021-11-14 22:31:04,313 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.LocalJobRunner - map
2021-11-14 22:31:04,314 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.Task - Task 'attempt_local2048462731_0001_m_000000_0'
done.
2021-11-14 22:31:04,314 [LocalJobRunner Map Task Executor #0] INFO
org.apache.hadoop.mapred.LocalJobRunner - Finishing task:
attempt_local2048462731_0001_m_000000_0
2021-11-14 22:31:04,314 [Thread-7] INFO org.apache.hadoop.mapred.LocalJobRunner -
map task executor complete.
2021-11-14 22:31:04,316 [Thread-7] INFO org.apache.hadoop.mapred.LocalJobRunner -
Waiting for reduce tasks
2021-11-14 22:31:04,316 [pool-3-thread-1] INFO
org.apache.hadoop.mapred.LocalJobRunner - Starting task:
attempt_local2048462731_0001_r_000000_0
2021-11-14 22:31:04,362 [pool-3-thread-1] INFO
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer
Algorithm version is 1
2021-11-14 22:31:04,362 [pool-3-thread-1] INFO
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - FileOutputCommitter
skip cleanup _temporary folders under output directory:false, ignore cleanup
failures: false
2021-11-14 22:31:04,364 [pool-3-thread-1] INFO org.apache.hadoop.mapred.Task -
Using ResourceCalculatorProcessTree : [ ]
2021-11-14 22:31:04,371 [pool-3-thread-1] INFO org.apache.hadoop.mapred.ReduceTask
- Using ShuffleConsumerPlugin:
org.apache.hadoop.mapreduce.task.reduce.Shuffle@34ddeada
2021-11-14 22:31:04,416 [pool-3-thread-1] INFO
org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl - MergerManager:
memoryLimit=709551680, maxSingleShuffleLimit=177387920, mergeThreshold=468304128,
ioSortFactor=10, memToMemMergeOutputsThreshold=10
2021-11-14 22:31:04,425 [EventFetcher for fetching Map Completion Events] INFO
org.apache.hadoop.mapreduce.task.reduce.EventFetcher -
attempt_local2048462731_0001_r_000000_0 Thread started: EventFetcher for fetching
Map Completion Events
2021-11-14 22:31:04,500 [localfetcher#1] INFO
org.apache.hadoop.mapreduce.task.reduce.LocalFetcher - localfetcher#1 about to
shuffle output of map attempt_local2048462731_0001_m_000000_0 decomp: 77 len: 81 to
MEMORY
2021-11-14 22:31:04,516 [localfetcher#1] INFO
org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput - Read 77 bytes from map-
output for attempt_local2048462731_0001_m_000000_0
2021-11-14 22:31:04,517 [localfetcher#1] INFO
org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl - closeInMemoryFile ->
map-output of size: 77, inMemoryMapOutputs.size() -> 1, commitMemory -> 0,
usedMemory ->77
2021-11-14 22:31:04,521 [Readahead Thread #0] WARN
org.apache.hadoop.io.ReadaheadPool - Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)
at
org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:2
67)
at
org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossibl
e(NativeIO.java:146)
at
org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2021-11-14 22:31:04,524 [EventFetcher for fetching Map Completion Events] INFO
org.apache.hadoop.mapreduce.task.reduce.EventFetcher - EventFetcher is
interrupted.. Returning
2021-11-14 22:31:04,525 [pool-3-thread-1] INFO
org.apache.hadoop.mapred.LocalJobRunner - 1 / 1 copied.
2021-11-14 22:31:04,526 [pool-3-thread-1] INFO
org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl - finalMerge called with 1
in-memory map-outputs and 0 on-disk map-outputs
2021-11-14 22:31:04,552 [pool-3-thread-1] INFO org.apache.hadoop.mapred.Merger -
Merging 1 sorted segments
2021-11-14 22:31:04,552 [pool-3-thread-1] INFO org.apache.hadoop.mapred.Merger -
Down to the last merge-pass, with 1 segments left of total size: 68 bytes
2021-11-14 22:31:04,554 [pool-3-thread-1] INFO
org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl - Merged 1 segments, 77
bytes to disk to satisfy reduce memory limit
2021-11-14 22:31:04,554 [pool-3-thread-1] INFO
org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl - Merging 1 files, 81
bytes from disk
2021-11-14 22:31:04,556 [pool-3-thread-1] INFO
org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl - Merging 0 segments, 0
bytes from memory into reduce
2021-11-14 22:31:04,556 [pool-3-thread-1] INFO org.apache.hadoop.mapred.Merger -
Merging 1 sorted segments
2021-11-14 22:31:04,557 [pool-3-thread-1] INFO org.apache.hadoop.mapred.Merger -
Down to the last merge-pass, with 1 segments left of total size: 68 bytes
2021-11-14 22:31:04,558 [pool-3-thread-1] INFO
org.apache.hadoop.mapred.LocalJobRunner - 1 / 1 copied.
2021-11-14 22:31:04,569 [pool-3-thread-1] INFO
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer
Algorithm version is 1
2021-11-14 22:31:04,569 [pool-3-thread-1] INFO
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - FileOutputCommitter
skip cleanup _temporary folders under output directory:false, ignore cleanup
failures: false
2021-11-14 22:31:04,574 [pool-3-thread-1] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.skip.on is deprecated.
Instead, use mapreduce.job.skiprecords
2021-11-14 22:31:04,596 [pool-3-thread-1] WARN
org.apache.pig.data.SchemaTupleBackend - SchemaTupleBackend has already been
initialized
2021-11-14 22:31:04,619 [pool-3-thread-1] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce -
Aliases being processed per job phase (AliasName[line,offset]): M:
student_info[1,15],student_info[-1,-1],citywise_avg_age[5,19],citywise[3,11] C:
citywise_avg_age[5,19],citywise[3,11] R: citywise_avg_age[5,19]
2021-11-14 22:31:04,631 [pool-3-thread-1] INFO org.apache.hadoop.mapred.Task -
Task:attempt_local2048462731_0001_r_000000_0 is done. And is in the process of
committing
2021-11-14 22:31:04,635 [pool-3-thread-1] INFO
org.apache.hadoop.mapred.LocalJobRunner - 1 / 1 copied.
2021-11-14 22:31:04,635 [pool-3-thread-1] INFO org.apache.hadoop.mapred.Task -
Task attempt_local2048462731_0001_r_000000_0 is allowed to commit now
2021-11-14 22:31:04,639 [pool-3-thread-1] INFO
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter - Saved output of task
'attempt_local2048462731_0001_r_000000_0' to
file:/home/cloudera/pig/city_avg_age1/_temporary/0/task_local2048462731_0001_r_0000
00
2021-11-14 22:31:04,644 [pool-3-thread-1] INFO
org.apache.hadoop.mapred.LocalJobRunner - reduce > reduce
2021-11-14 22:31:04,644 [pool-3-thread-1] INFO org.apache.hadoop.mapred.Task -
Task 'attempt_local2048462731_0001_r_000000_0' done.
2021-11-14 22:31:04,645 [pool-3-thread-1] INFO
org.apache.hadoop.mapred.LocalJobRunner - Finishing task:
attempt_local2048462731_0001_r_000000_0
2021-11-14 22:31:04,645 [Thread-7] INFO org.apache.hadoop.mapred.LocalJobRunner -
reduce task executor complete.
2021-11-14 22:31:09,574 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.map.tasks is deprecated.
Instead, use mapreduce.job.maps
2021-11-14 22:31:15,577 [main] WARN org.apache.pig.tools.pigstats.PigStatsUtil -
Failed to get RunningJob for job job_local2048462731_0001
2021-11-14 22:31:15,586 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher -
100% complete
2021-11-14 22:31:15,587 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats -
Detected Local mode. Stats reported below may be incomplete
2021-11-14 22:31:15,610 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats -
Script Statistics:

HadoopVersion PigVersion UserId StartedAt FinishedAt Features

2.6.0-cdh5.12.0 0.12.0-cdh5.12.0 cloudera 2021-11-14 22:31:02 2021-11-14
22:31:15 GROUP_BY

Success!

Job Stats (time in seconds):

JobId Alias Feature Outputs
job_local2048462731_0001 citywise,citywise_avg_age,student_info
GROUP_BY,COMBINER /home/cloudera/pig/city_avg_age1,

Input(s):
Successfully read records from: "/home/cloudera/pig/sample_student_data.txt"

Output(s):
Successfully stored records in: "/home/cloudera/pig/city_avg_age1"

Job DAG:
job_local2048462731_0001

2021-11-14 22:31:21,613 [main] INFO

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher -
Success!

[cloudera@quickstart ~]$ ls pig/city_avg_age1

part-r-00000 _SUCCESS
[cloudera@quickstart ~]$ cat ^C
[cloudera@quickstart ~]$ cat part-r-00000
cat: part-r-00000: No such file or directory
[cloudera@quickstart ~]$ cat /home/cloudera/pig/city_avg_age1/part-r-00000
Pune|21.0
Mumbai|22.0
Chennai|23.5
Kolkata|23.0

A = LOAD '/home/cloudera/pig/sample_student_data.txt' using PigStorage(',') AS

(id:chararray, fname:chararray, lname:chararray, age:int, phone:chararray,
city:chararray);