Hadoop Day4

(foremost)
(role)
(that)
(mobile)
(phones)
(play)
(in)
(our)
(lives)
(is)
(that)
(they)
(provide)
(us)
(an)
(easy)
(and)
(fast)
(way)
(of)
(communication.)
(Go)
(wherever)
(you)
(want)
(whenever)
(you)
(want;)
(your)
(mobile)
(phone)
(will)
(stay)
(you)
(connected)
(with)
(all.)
(Got)
(an)
(important)
(message)
(for)
(your)
(friend/family)
(member?)
(Just)
(send)
(instant)
(SMS)
(or)
(call)
(straight)
(away)
(and)
(make)
(your)
(important)
(message)
(reach)
(the)
(intended)
(person)
(in)
(no)
(time.)
(Simple!)
()
(Internet)
(Access)
(&)
(Social)
(Media)
()
(The)
(Internet)
(is)
(one)
(of)
(the)
(biggest)
(blessings)
(to)
(man)
(by)
(technology.)
(One)
(just)
(cannot)
(imagine)
(the)
(life)
(without)
(the)
(Internet.)
(Everyone)
(likes/wants)
(to)
(stay)
(connected)
(with)
(the)
(Internet)
(all)
(the)
(time.)
(Thanks)
(to)
(the)
(mobile)
(phones)
(that)
(make)
(it)
(possible.)
(The)
(mobile)
(phones)
(let)
(users)
(enjoy)
(the)
(social)
(media)
(on)
(the)
(go.)
(A)
(major)
(part)
(of)
(modern)
(world)
(is)
(addicted)
(to)
(social)
(networking)
(sites.)
(Mobile)
(Phones)
(let)
(them)
(be)
(active)
(in)
(social)
(media)
(on)
(the)
(move.)
()
(Mobile)
(Phones)
(Save)
(Money)
()
(Yes)
(they)
(do.)
(Just)
(buy)
(mid-range)
(or)
(high-end)
(mobile)
(phone)
(and)
(ultimately)
(you)
(become)
(proud)
(owner)
(of)
(many)
(gadgets)
(the)
(gadgets)
(that)
(are)
(packed)
(in)
(the)
(mobile)
(phone.)
(For)
(example)
(by)
(buying)
(mobile)
(phone)
(you)
(don t)
(need)
(to)
(spend)
(money)
(on)
(a)
(home)
(phone)
(you)
(don t)
(need)
(to)
(buy)
(any)
(music)
(or)
(video)
(player)
(you)
(don t)
(need)
(to)
(buy)
(radio)
(etc.)
(If)
(the)
(mobile)
(phone)
(is)
(high-end)
(then)
(you)
(need)
(not)
(to)
(buy)
(gaming)
(console)
(and)
(maybe)
(a)
(computer)
(as)
(well.)
()
(Mobile)
(Phones)
(Ensure)
(Safety)
()
(The)
(mobile)
(phones)
(provide)
(us)
(security.)
(How?)
(Well)
(to)
(understand)
(this)
(better)
(just)
(consider)
(any)
(situation)
(in)
(which)
(you)
(urgently)
(need)
(to)
(contact)
(fire)
(brigade)
(or)
(police.)
(What)
(would)
(you)
(do)
(in)
(such)
(case?)
(The)
(landlines)
(are)
(not)
(available)
(everywhere)
(but)
(the)
(mobile)
(phones)
(do.)
(Just)
(take)
(your)
(mobile)
(phone)
(out)
(dial)
(the)
(number)
(and)
(that s)
(it.)
(Your)
(one)
(good)
(step)
(of)
(buying)
(mobile)
(phone)
(will)
(prove)
(as)
(great)
(money)
(saver)
(for)
(you.)
()
(Mobile)
(Phones)
(Helps)
(in)
(Business)
()
(Got)
(an)
(important)
(message)
(for)
(your)
(client)
(or)
(employee?)
(Pick)
(up)
(your)
(mobile)
(phone)
(and)
(communicate)
(with)
(the)
(intended)
(person)
(straight)
(away.)
(Since)
(businesses)
(require)
(constant)
(communication)
(so)
(one)
(just)
(can t)
(imagine)
(any)
(business)
(without)
(the)
(use)
(of)
(mobile)
(phones.)
(And)
(with)
(all)
(those)
(new)
(smart)
(phones)
(available)
(in)
(market)
(these)
(days)
(a)
(businessman)
(can)
(organize)
(the)
(schedule)
(and)
(set)
(reminders)
(so)
(he)
(does)
(not)
(forget)
(any)
(important)
(meeting.)
(Mobile)
(phones)
(come)
(packed)
(with)
(lot)
(of)
(business)
(apps)
(which)
(makes)
(their)
(life)
(easier.)
(Thus)
(mobile)
(phones)
(are)
(like)
(a)
(boon)
(for)
(businesses.)
()
(So)
(folks!)
(These)
(are)
(the)
(main)
(benefits)
(of)
(having)
(mobile)
(phones)
(in)
(our)
(life.)
(Now)
(it s)
(your)
(turn)
(tell)
(us)
(why)
(Mobile)
(phone)
(is)
(important)
(for)
(you)
(in)
(the)
(comments)
(below.)
()
(More:)
(Positive)
(&)
(negative)
(impact)
(of)
(mobile)
()
()
()
(Read)
(more:)
(http://myessaypoint.com/importance-of-mobile-phone-technology#ixzz43FAnUmFx)
()
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt> group1 = group words1 by $0;
2016-03-19 07:02:57,041 [main] WARN org.apache.pig.newplan.BaseOperatorPlan ncountered Warning USING_OVERLOADED_FUNCTION 1 time(s).
2016-03-19 07:02:57,041 [main] WARN org.apache.pig.newplan.BaseOperatorPlan ncountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s).
grunt> dump group1
2016-03-19 07:03:07,786 [main] WARN org.apache.pig.newplan.BaseOperatorPlan ncountered Warning USING_OVERLOADED_FUNCTION 1 time(s).
2016-03-19 07:03:07,786 [main] WARN org.apache.pig.newplan.BaseOperatorPlan ncountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s).
E
E
E
E
2016-03-19 07:03:07,787 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: GROUP_BY
2016-03-19 07:03:07,809 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 07:03:07,809 [main] INFO org.apache.pig.data.SchemaTupleBackend - Ke
y [pig.schematuple] was not set... will not generate code.
2016-03-19 07:03:07,809 [main] INFO org.apache.pig.newplan.logical.optimizer.Lo
gicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalc
ulator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeF
ilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushD
ownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2016-03-19 07:03:07,813 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? fal
se
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2016-03-19 07:03:07,824 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:03:07,827 [main] INFO org.apache.pig.tools.pigstats.mapreduce.MRS
criptState - Pig script settings are added to the job
ne.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percen
t is not set, set to default 0.3
ne.mapReduceLayer.JobControlCompiler - Reduce phase detected, estimating # of re
quired reducers.
ne.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.b
ackend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
ne.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxRedu
cers=999 totalInputFileSize=3386
ne.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
ne.mapReduceLayer.JobControlCompiler - Setting up single store job
2016-03-19 07:03:07,854 [main] INFO org.apache.pig.data.SchemaTupleFrontend - K
ey [pig.schematuple] is false, will not generate code.
2016-03-19 07:03:07,854 [main] INFO org.apache.pig.data.SchemaTupleFrontend - S
tarting process to move generated code to distributed cacche
2016-03-19 07:03:07,854 [main] INFO org.apache.pig.data.SchemaTupleFrontend - D
istributed cache not supported or needed in local mode. Setting key [pig.schemat
uple.local.dir] with code temp directory: /tmp/1458370987854-0
ne.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission
.
2016-03-19 07:03:07,888 [JobControl] INFO org.apache.hadoop.metrics.jvm.JvmMetr
ics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - al
ready initialized
2016-03-19 07:03:07,894 [JobControl] WARN org.apache.hadoop.mapreduce.JobResour
ceUploader - No job jar file set. User classes may not be found. See Job or Job
#setJar(String).
2016-03-19 07:03:07,903 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input
.FileInputFormat - Total input paths to process : 1
2016-03-19 07:03:07,905 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths to process : 1
onengine.util.MapRedUtil - Total input paths (combined) to process : 1
2016-03-19 07:03:07,907 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - number of splits:1
ter - Submitting tokens for job: job_local1118178464_0005
2016-03-19 07:03:08,030 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The
url to track the job: http://localhost:8080/
2016-03-19 07:03:08,031 [Thread-27] INFO org.apache.hadoop.mapred.LocalJobRunne
r - OutputCommitter set in config null
2016-03-19 07:03:08,036 [Thread-27] INFO org.apache.hadoop.conf.Configuration.d
eprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
eprecation - mapred.job.reduce.markreset.buffer.percent is deprecated. Instead,
use mapreduce.reduce.markreset.buffer.percent
eprecation - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduc
es
eprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker
.address
eprecation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-che
cksum
2016-03-19 07:03:08,036 [Thread-27] INFO org.apache.hadoop.mapreduce.lib.output
.FileOutputCommitter - File Output Committer Algorithm version is 1
r - OutputCommitter is org.apache.pig.backend.hadoop.executionengine.mapReduceLa
yer.PigOutputCommitter
r - Waiting for map tasks
2016-03-19 07:03:08,039 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Starting task: attempt_local1118178464_0005_m_0000
00_0
adoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer Algorithm
version is 1
adoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ]
adoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 3386
Input split[0]:
Length = 3386
ClassName: org.apache.hadoop.mapreduce.lib.input.FileSplit
Locations:
----------------------2016-03-19 07:03:08,050 [LocalJobRunner Map Task Executor #0] INFO
ig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader being processed file:/root/essay:0+3386
2016-03-19 07:03:08,124 [LocalJobRunner Map Task Executor #0] INFO
adoop.mapred.MapTask - (EQUATOR) 0 kvi 26214396(104857584)
org.apache.p
Current split
org.apache.h
org.apache.h
adoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100

adoop.mapred.MapTask - soft limit at 83886080
adoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600
adoop.mapred.MapTask - kvstart = 26214396; length = 6553600
adoop.mapred.MapTask - Map output collector class = org.apache.hadoop.mapred.Map
Task$MapOutputBuffer
2016-03-19 07:03:08,129 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not gener
ate code.
ig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map - Alias
es being processed per job phase (AliasName[line,offset]): M: input1[4,9],words1
[-1,-1],group1[7,9] C: R:
adoop.mapred.LocalJobRunner 2016-03-19 07:03:08,164 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Starting flush of map output
adoop.mapred.MapTask - Spilling map output
adoop.mapred.MapTask - bufstart = 0; bufend = 6318; bufvoid = 104857600
adoop.mapred.MapTask - kvstart = 26214396(104857584); kvend = 26212024(104848096
); length = 2373/6553600
adoop.mapred.MapTask - Finished spill 0
adoop.mapred.Task - Task:attempt_local1118178464_0005_m_000000_0 is done. And is
in the process of committing
adoop.mapred.LocalJobRunner - map
adoop.mapred.Task - Task 'attempt_local1118178464_0005_m_000000_0' done.
adoop.mapred.LocalJobRunner - Finishing task: attempt_local1118178464_0005_m_000
000_0
r - map task executor complete.
r - Waiting for reduce tasks
2016-03-19 07:03:08,214 [pool-14-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - Starting task: attempt_local1118178464_0005_r_000000_0
2016-03-19 07:03:08,221 [pool-14-thread-1] INFO org.apache.hadoop.mapreduce.lib
.output.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 07:03:08,227 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Task Using ResourceCalculatorProcessTree : [ ]
2016-03-19 07:03:08,227 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Reduce
Task - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuf
fle@6ab6fc14
2016-03-19 07:03:08,228 [pool-14-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - MergerManager: memoryLimit=709551680, maxSingleShuff
leLimit=177387920, mergeThreshold=468304128, ioSortFactor=10, memToMemMergeOutpu
tsThreshold=10
2016-03-19 07:03:08,229 [EventFetcher for fetching Map Completion Events] INFO
org.apache.hadoop.mapreduce.task.reduce.EventFetcher - attempt_local1118178464_0
005_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2016-03-19 07:03:08,231 [localfetcher#2] INFO org.apache.hadoop.mapreduce.task.

reduce.LocalFetcher - localfetcher#2 about to shuffle output of map attempt_loca
l1118178464_0005_m_000000_0 decomp: 7508 len: 7512 to MEMORY
reduce.InMemoryMapOutput - Read 7508 bytes from map-output for attempt_local1118
178464_0005_m_000000_0
reduce.MergeManagerImpl - closeInMemoryFile -> map-output of size: 7508, inMemor
yMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->7508
org.apache.hadoop.mapreduce.task.reduce.EventFetcher - EventFetcher is interrupt
ed.. Returning
obRunner - 1 / 1 copied.
k.reduce.MergeManagerImpl - finalMerge called with 1 in-memory map-outputs and 0
on-disk map-outputs
2016-03-19 07:03:08,236 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Merger
- Merging 1 sorted segments
- Down to the last merge-pass, with 1 segments left of total size: 7502 bytes
2016-03-19 07:03:08,261 [Readahead Thread #3] WARN org.apache.hadoop.io.Readahe
adPool - Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Met
hod)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(N
ativeIO.java:267)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFa
dviseIfPossible(NativeIO.java:146)
at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(Readahead
Pool.java:206)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
at java.lang.Thread.run(Thread.java:745)
k.reduce.MergeManagerImpl - Merged 1 segments, 7508 bytes to disk to satisfy red
uce memory limit
k.reduce.MergeManagerImpl - Merging 1 files, 7512 bytes from disk
k.reduce.MergeManagerImpl - Merging 0 segments, 0 bytes from memory into reduce
2016-03-19 07:03:08,274 [pool-14-thread-1] WARN org.apache.pig.data.SchemaTuple
Backend - SchemaTupleBackend has already been initialized
2016-03-19 07:03:08,276 [pool-14-thread-1] INFO org.apache.pig.backend.hadoop.e
xecutionengine.mapReduceLayer.PigMapReduce$Reduce - Aliases being processed per
job phase (AliasName[line,offset]): M: input1[4,9],words1[-1,-1],group1[7,9] C:
R:
2016-03-19 07:03:08,349 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Task Task:attempt_local1118178464_0005_r_000000_0 is done. And is in the process of
committing
2016-03-19 07:03:08,351 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Task Task attempt_local1118178464_0005_r_000000_0 is allowed to commit now
.output.FileOutputCommitter - Saved output of task 'attempt_local1118178464_0005
_r_000000_0' to file:/tmp/temp2073304142/tmp664834969/_temporary/0/task_local111
8178464_0005_r_000000
obRunner - reduce > reduce
2016-03-19 07:03:08,354 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Task Task 'attempt_local1118178464_0005_r_000000_0' done.
obRunner - Finishing task: attempt_local1118178464_0005_r_000000_0
r - reduce task executor complete.
ne.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local1118178464_0005
ne.mapReduceLayer.MapReduceLauncher - Processing aliases group1,input1,words1
ne.mapReduceLayer.MapReduceLauncher - detailed locations: M: input1[4,9],words1[
-1,-1],group1[7,9] C: R:
ne.mapReduceLayer.MapReduceLauncher - Running jobs are [job_local1118178464_0005
]
initialized
initialized
initialized
ne.mapReduceLayer.MapReduceLauncher - 100% complete
2016-03-19 07:03:08,542 [main] INFO org.apache.pig.tools.pigstats.mapreduce.Sim
plePigStats - Script Statistics:
HadoopVersion PigVersion
UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 07:03:07
2016-03-19 07:03:08
Features
GROUP_BY
Success!
Job Stats (time in seconds):
JobId Maps
Reduces MaxMapTime
MinMapTime
AvgMapTime
MedianMa
pTime MaxReduceTime MinReduceTime AvgReduceTimeMedianReducetime Alias Fe
ature Outputs
job_local1118178464_0005
1
1
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
group1,input1,words1 GROUP_BY file:/tmp/temp2073
304142/tmp664834969,
Input(s):
Successfully read 30 records from: "/root/essay"
Output(s):
Successfully stored 296 records in: "file:/tmp/temp2073304142/tmp664834969"
Counters:
Total records written : 296
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_local1118178464_0005
initialized
initialized
initialized
ne.mapReduceLayer.MapReduceLauncher - Success!
2016-03-19 07:03:08,554 [main] WARN org.apache.pig.data.SchemaTupleBackend - Sc
hemaTupleBackend has already been initialized
2016-03-19 07:03:08,572 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileI
nputFormat - Total input paths to process : 1
ne.util.MapRedUtil - Total input paths to process : 1
(&,{(&),(&)})
(A,{(A)})
(a,{(a),(a),(a),(a),(a),(a),(a),(a),(a),(a),(a)})
(Go,{(Go)})
(If,{(If)})
(In,{(In)})
(So,{(So),(So)})
(an,{(an),(an),(an)})
(as,{(as),(as),(as),(as)})
(be,{(be)})
(by,{(by),(by)})
(do,{(do)})
(he,{(he)})
(in,{(in),(in),(in),(in),(in),(in),(in),(in),(in),(in),(in),(in)})
(is,{(is),(is),(is),(is),(is),(is),(is),(is),(is)})
(it,{(it),(it)})
(no,{(no)})
(of,{(of),(of),(of),(of),(of),(of),(of),(of),(of),(of),(of),(of),(of),(of),(of)}
)
(on,{(on),(on),(on)})
(or,{(or),(or),(or),(or),(or)})
(so,{(so),(so)})
(to,{(to),(to),(to),(to),(to),(to),(to),(to),(to),(to),(to),(to),(to)})
(up,{(up)})
(us,{(us),(us),(us)})
(And,{(And)})
(But,{(But)})
(For,{(For)})
(Got,{(Got),(Got)})
(Now,{(Now)})
(One,{(One)})
(SMS,{(SMS)})
(The,{(The),(The),(The),(The),(The),(The)})
(Way,{(Way)})
(Yes,{(Yes)})
(all,{(all),(all)})
(and,{(and),(and),(and),(and),(and),(and),(and),(and),(and)})
(any,{(any),(any),(any),(any)})
(are,{(are),(are),(are),(are),(are),(are),(are)})
(big,{(big)})
(but,{(but)})
(buy,{(buy),(buy),(buy),(buy)})
(can,{(can)})
(do.,{(do.),(do.)})
(few,{(few)})
(for,{(for),(for),(for),(for),(for),(for)})
(go.,{(go.)})
(has,{(has)})
(it.,{(it.)})
(let,{(let),(let)})
(lot,{(lot)})
(man,{(man)})
(new,{(new)})
(not,{(not),(not),(not),(not),(not)})
(one,{(one),(one),(one)})
(our,{(our),(our),(our)})
(out,{(out)})
(own,{(own)})
(set,{(set)})
(the,{(the),(the),(the),(the),(the),(the),(the),(the),(the),(the),(the),(the),(t
he),(the),(the),(the),(the),(the),(the),(the),(the),(the),(the),(the),(the),(the
),(the)})
(use,{(use)})
(way,{(way)})
(who,{(who)})
(why,{(why),(why)})
(you,{(you),(you),(you),(you),(you),(you),(you),(you),(you),(you),(you),(you),(y
ou)})
(Easy,{(Easy)})
(Gone,{(Gone)})
(Here,{(Here)})
(How?,{(How?)})
(Just,{(Just),(Just),(Just),(Just)})
(Pick,{(Pick)})
(Read,{(Read)})
(Save,{(Save)})
(Thus,{(Thus)})
(Well,{(Well)})
(What,{(What),(What),(What)})
(Yeah,{(Yeah)})
(Your,{(Your)})
(all.,{(all.)})
(apps,{(apps)})
(away,{(away)})
(boon,{(boon)})
(call,{(call)})
(come,{(come)})
(days,{(days),(days)})
(dial,{(dial)})
(does,{(does),(does)})
(easy,{(easy)})
(etc.,{(etc.)})
(fast,{(fast)})
(find,{(find)})
(fire,{(fire)})
(for?,{(for?)})
(good,{(good)})
(hard,{(hard)})
(have,{(have)})
(home,{(home)})
(just,{(just),(just),(just)})
(life,{(life),(life)})
(like,{(like)})
(main,{(main)})
(make,{(make),(make)})
(many,{(many)})
(need,{(need),(need),(need),(need),(need)})
(part,{(part)})
(play,{(play)})
(role,{(role)})
(send,{(send)})
(stay,{(stay),(stay)})
(step,{(step)})
(such,{(such)})
(tags,{(tags)})
(take,{(take)})
(tell,{(tell)})
(that,{(that),(that),(that),(that),(that),(that)})
(them,{(them)})
(then,{(then)})
(they,{(they),(they)})
(this,{(this)})
(time,{(time)})
(turn,{(turn)})
(very,{(very)})
(want,{(want)})
(were,{(were)})
(when,{(when)})
(will,{(will),(will)})
(with,{(with),(with),(with),(with),(with)})
(you.,{(you.)})
(your,{(your),(your),(your),(your),(your),(your),(your)})
(Helps,{(Helps)})
(Media,{(Media)})
(Money,{(Money)})
(More:,{(More:)})
(Since,{(Since)})
(These,{(These)})
(away.,{(away.)})
(basic,{(basic),(basic)})
(bucks,{(bucks)})
(case?,{(case?)})
(deal.,{(deal.)})
(enjoy,{(enjoy)})
(first,{(first)})
(great,{(great)})
(life.,{(life.),(life.)})
(life?,{(life?)})
(lives,{(lives)})
(major,{(major)})
(makes,{(makes)})
(maybe,{(maybe)})
(meant,{(meant)})
(media,{(media),(media)})
(money,{(money),(money)})
(more:,{(more:)})
(move.,{(move.)})
(music,{(music)})
(owner,{(owner),(owner)})
(phone,{(phone),(phone),(phone),(phone),(phone),(phone),(phone),(phone),(phone),
(phone)})
(proud,{(proud),(proud)})
(prove,{(prove)})
(radio,{(radio)})
(reach,{(reach)})
(saver,{(saver)})
(small,{(small)})
(smart,{(smart)})
(spend,{(spend),(spend)})
(their,{(their)})
(these,{(these)})
(thing,{(thing)})
(those,{(those)})
(time.,{(time.),(time.)})
(users,{(users)})
(video,{(video)})
(want;,{(want;)})
(well.,{(well.)})
(which,{(which),(which)})
(with.,{(with.)})
(world,{(world)})
(would,{(would)})
(Access,{(Access)})
(Ensure,{(Ensure)})
(Mobile,{(Mobile),(Mobile),(Mobile),(Mobile),(Mobile),(Mobile)})
(Phones,{(Phones),(Phones),(Phones),(Phones)})
(Safety,{(Safety)})
(Social,{(Social)})
(Thanks,{(Thanks)})
(active,{(active)})
(become,{(become)})
(below.,{(below.)})
(better,{(better)})
(buying,{(buying),(buying),(buying)})
(cannot,{(cannot)})
(client,{(client)})
(extent,{(extent)})
(folks!,{(folks!)})
(forget,{(forget)})
(gadget,{(gadget)})
(gaming,{(gaming)})
(having,{(having)})
(impact,{(impact)})
(it s,{(it s),(it s)})
(lives?,{(lives?)})
(luxury,{(luxury)})
(market,{(market)})
(mobile,{(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile)
,(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile
),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobil
e)})
(modern,{(modern)})
(number,{(number)})
(packed,{(packed),(packed)})
(person,{(person),(person),(person)})
(phone.,{(phone.),(phone.),(phone.)})
(phones,{(phones),(phones),(phones),(phones),(phones),(phones),(phones),(phones)
,(phones),(phones),(phones),(phones),(phones),(phones)})
(player,{(player)})
(prices,{(prices)})
(right-,{(right-)})
(sites.,{(sites.)})
(social,{(social),(social),(social)})
(Simple!,{(Simple!)})
(answer.,{(answer.)})
(biggest,{(biggest)})
(brigade,{(brigade)})
(can t,{(can t)})
(console,{(console)})
(contact,{(contact)})
(don t,{(don t),(don t),(don t)})
(easier.,{(easier.)})
(example,{(example)})
(gadgets,{(gadgets),(gadgets)})
(growing,{(growing)})
(imagine,{(imagine),(imagine)})
(instant,{(instant)})
(lowered,{(lowered)})
(member?,{(member?)})
(message,{(message),(message),(message)})
(phones.,{(phones.)})
(police.,{(police.)})
(provide,{(provide),(provide)})
(require,{(require)})
(without,{(without),(without)})
(Business,{(Business)})
(Everyone,{(Everyone)})
(Internet,{(Internet),(Internet),(Internet)})
(Positive,{(Positive)})
(addicted,{(addicted)})
(answered,{(answered)})
(benefits,{(benefits)})
(business,{(business),(business)})
(comments,{(comments)})
(computer,{(computer)})
(consider,{(consider)})
(constant,{(constant)})
(everyone,{(everyone)})
(foremost,{(foremost)})
(high-end,{(high-end),(high-end)})
(intended,{(intended),(intended)})
(meeting.,{(meeting.)})
(negative,{(negative)})
(nowadays,{(nowadays)})
(organize,{(organize)})
(schedule,{(schedule)})
(straight,{(straight),(straight)})
(that s,{(that s)})
(urgently,{(urgently)})
(whenever,{(whenever)})
(wherever,{(wherever)})
(Internet.,{(Internet.)})
(available,{(available),(available)})
(blessings,{(blessings)})
(connected,{(connected),(connected)})
(employee?,{(employee?)})
(important,{(important),(important),(important),(important),(important)})
(landlines,{(landlines)})
(mid-range,{(mid-range)})
(necessity,{(necessity),(necessity)})
(possible.,{(possible.)})
(reminders,{(reminders)})
(security.,{(security.)})
(situation,{(situation)})
(today s,{(today s)})
(businesses,{(businesses)})
(considered,{(considered)})
(everywhere,{(everywhere)})
(importance,{(importance)})
(networking,{(networking)})
(ultimately,{(ultimately)})
(understand,{(understand)})
(businesses.,{(businesses.)})
(businessman,{(businessman)})
(communicate,{(communicate)})
(competition,{(competition)})
(likes/wants,{(likes/wants)})
(technology.,{(technology.)})
(Communication,{(Communication)})
(communication,{(communication)})
(friend/family,{(friend/family)})
(manufacturers,{(manufacturers)})
(communication.,{(communication.),(communication.)})
(http://myessaypoint.com/importance-of-mobile-phone-technology#ixzz43FAnUmFx,{(h
ttp://myessaypoint.com/importance-of-mobile-phone-technology#ixzz43FAnUmFx)})
(,{(),(),(),(),(),(),(),(),(),(),(),(),(),(),(),()})
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt> count1 = foreach group1 generate $0,count($1);
2016-03-19 07:05:19,883 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 10
70: Could not resolve count using imports: [, java.lang., org.apache.pig.builtin
., org.apache.pig.impl.builtin.]
Details at logfile: /root/pig_1458370235691.log
grunt> count1 = foreach group1 generate $0,COUNT($1);
2016-03-19 07:05:29,168 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - E
ncountered Warning USING_OVERLOADED_FUNCTION 1 time(s).
ncountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s).
grunt> DUMP count1

2016-03-19 07:05:39,923 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: GROUP_BY
se
ne.util.CombinerOptimizerUtil - Choosing to move algebraic foreach to combiner
initialized
quired reducers.
.

ready initialized
#setJar(String).
es
.address
cksum
0_0
version is 1
Total Length = 3386
Input split[0]:
Length = 3386
Locations:
----------------------2016-03-19 07:05:40,186 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader - Current split
being processed file:/root/essay:0+3386

ate code.
[-1,-1],count1[8,9],group1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9]
); length = 2373/6553600
ig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine - Aliases b
eing processed per job phase (AliasName[line,offset]): M: input1[4,9],words1[-1,
-1],count1[8,9],group1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9]
00_0
fle@5f81a1f3
tsThreshold=10
00243_0006_m_000000_0
ed.. Returning
on-disk map-outputs
hod)
ativeIO.java:267)
Pool.java:206)
java:1145)
.java:615)
uce memory limit

job phase (AliasName[line,offset]): M: input1[4,9],words1[-1,-1],count1[8,9],gro
up1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9]
ne.mapReduceLayer.MapReduceLauncher - Processing aliases count1,group1,input1,wo
rds1
-1,-1],count1[8,9],group1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9]
ne.mapReduceLayer.MapReduceLauncher - Running jobs are [job_local709000243_0006]
2016-03-19 07:05:40,602 [pool-17-thread-1] INFO org.apache.hadoop.mapred.Task Task:attempt_local709000243_0006_r_000000_0 is done. And is in the process of c
ommitting
.output.FileOutputCommitter - Saved output of task 'attempt_local709000243_0006_
r_000000_0' to file:/tmp/temp2073304142/tmp1910013485/_temporary/0/task_local709
000243_0006_r_000000
initialized
initialized
initialized
UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 07:05:40
2016-03-19 07:05:40
Features
GROUP_BY
Success!
JobId Maps
Reduces MaxMapTime
pTime MaxReduceTime MinReduceTime
ature Outputs
MinMapTime
AvgMapTime
MedianMa
AvgReduceTimeMedianReducetime Alias Fe
job_local709000243_0006 1
1
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
count1,group1,input1,words1 GROUP_BY,COMBINER
le:/tmp/temp2073304142/tmp1910013485,
fi
Input(s):
Output(s):
Counters:
Job DAG:
job_local709000243_0006
initialized
initialized
initialized
(&,2)
(A,1)
(a,11)
(Go,1)
(If,1)
(In,1)
(So,2)
(an,3)
(as,4)
(be,1)
(by,2)
(do,1)
(he,1)
(in,12)
(is,9)
(it,2)
(no,1)
(of,15)
(on,3)
(or,5)
(so,2)
(to,13)
(up,1)
(us,3)
(And,1)
(But,1)
(For,1)
(Got,2)
(Now,1)
(One,1)
(SMS,1)
(The,6)
(Way,1)
(Yes,1)
(all,2)
(and,9)
(any,4)
(are,7)
(big,1)
(but,1)
(buy,4)
(can,1)
(do.,2)
(few,1)
(for,6)
(go.,1)
(has,1)
(it.,1)
(let,2)
(lot,1)
(man,1)
(new,1)
(not,5)
(one,3)
(our,3)
(out,1)
(own,1)
(set,1)
(the,27)
(use,1)
(way,1)
(who,1)
(why,2)
(you,13)
(Easy,1)
(Gone,1)
(Here,1)
(How?,1)
(Just,4)
(Pick,1)
(Read,1)
(Save,1)
(Thus,1)
(Well,1)
(What,3)
(Yeah,1)
(Your,1)
(all.,1)
(apps,1)
(away,1)
(boon,1)
(call,1)
(come,1)
(days,2)
(dial,1)
(does,2)
(easy,1)
(etc.,1)
(fast,1)
(find,1)
(fire,1)
(for?,1)
(good,1)
(hard,1)
(have,1)
(home,1)
(just,3)
(life,2)
(like,1)
(main,1)
(make,2)
(many,1)
(need,5)
(part,1)
(play,1)
(role,1)
(send,1)
(stay,2)
(step,1)
(such,1)
(tags,1)
(take,1)
(tell,1)
(that,6)
(them,1)
(then,1)
(they,2)
(this,1)
(time,1)
(turn,1)
(very,1)
(want,1)
(were,1)
(when,1)
(will,2)
(with,5)
(you.,1)
(your,7)
(Helps,1)
(Media,1)
(Money,1)
(More:,1)
(Since,1)
(These,1)
(away.,1)
(basic,2)
(bucks,1)
(case?,1)
(deal.,1)
(enjoy,1)
(first,1)
(great,1)
(life.,2)
(life?,1)
(lives,1)
(major,1)
(makes,1)
(maybe,1)
(meant,1)
(media,2)
(money,2)
(more:,1)
(move.,1)
(music,1)
(owner,2)
(phone,10)
(proud,2)
(prove,1)
(radio,1)
(reach,1)
(saver,1)
(small,1)
(smart,1)
(spend,2)
(their,1)
(these,1)
(thing,1)
(those,1)
(time.,2)
(users,1)
(video,1)
(want;,1)
(well.,1)
(which,2)
(with.,1)
(world,1)
(would,1)
(Access,1)
(Ensure,1)
(Mobile,6)
(Phones,4)
(Safety,1)
(Social,1)
(Thanks,1)
(active,1)
(become,1)
(below.,1)
(better,1)
(buying,3)
(cannot,1)
(client,1)
(extent,1)
(folks!,1)
(forget,1)
(gadget,1)
(gaming,1)
(having,1)
(impact,1)
(it s,2)
(lives?,1)
(luxury,1)
(market,1)
(mobile,26)
(modern,1)
(number,1)
(packed,2)
(person,3)
(phone.,3)
(phones,14)
(player,1)
(prices,1)
(right-,1)
(sites.,1)
(social,3)
(Simple!,1)
(answer.,1)
(biggest,1)
(brigade,1)
(can t,1)
(console,1)
(contact,1)
(don t,3)
(easier.,1)
(example,1)
(gadgets,2)
(growing,1)
(imagine,2)
(instant,1)
(lowered,1)
(member?,1)
(message,3)
(phones.,1)
(police.,1)
(provide,2)
(require,1)
(without,2)
(Business,1)
(Everyone,1)
(Internet,3)
(Positive,1)
(addicted,1)
(answered,1)
(benefits,1)
(business,2)
(comments,1)
(computer,1)
(consider,1)
(constant,1)
(everyone,1)
(foremost,1)
(high-end,2)
(intended,2)
(meeting.,1)
(negative,1)
(nowadays,1)
(organize,1)
(schedule,1)
(straight,2)
(that s,1)
(urgently,1)
(whenever,1)
(wherever,1)
(Internet.,1)
(available,2)
(blessings,1)
(connected,2)
(employee?,1)
(important,5)
(landlines,1)
(mid-range,1)
(necessity,2)
(possible.,1)
(reminders,1)
(security.,1)
(situation,1)
(today s,1)
(businesses,1)
(considered,1)
(everywhere,1)
(importance,1)
(networking,1)
(ultimately,1)
(understand,1)
(businesses.,1)
(businessman,1)
(communicate,1)
(competition,1)
(likes/wants,1)
(technology.,1)
(Communication,1)
(communication,1)
(friend/family,1)
(manufacturers,1)
(communication.,2)
(http://myessaypoint.com/importance-of-mobile-phone-technology#ixzz43FAnUmFx,1)
(,0)
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt> largest = Group count1 as $1>2;
00: <line 9, column 23> Syntax error, unexpected symbol at or near 'as'
grunt> largest = GROUP count1 as $1>2;
grunt> largest = foreach count1 as $1>2;
grunt> largest1 = foreach count1 as $1>2;
grunt> largest1 = filter count1 as $1>2;

2016-03-19 07:20:12,577 [main] ERROR org.apache.pig.tools.grunt.Grunt 00: <line 9, column 25> mismatched input 'as' expecting BY
grunt> largest1 = FILTER count1 as $1>2;
2016-03-19 07:20:26,255 [main] ERROR org.apache.pig.tools.grunt.Grunt 00: <line 9, column 25> mismatched input 'as' expecting BY
grunt> largest1 = FILTER count1 AS $1>2;
2016-03-19 07:20:48,791 [main] ERROR org.apache.pig.tools.grunt.Grunt 00: <line 9, column 25> mismatched input 'AS' expecting BY
grunt> largest1 = FILTER count1 $1>2;
2016-03-19 07:21:03,917 [main] ERROR org.apache.pig.tools.grunt.Grunt 00: <line 9, column 25> mismatched input '$1' expecting BY
grunt>
grunt> \
2016-03-19 07:22:25,550 [main] ERROR org.apache.pig.tools.grunt.Grunt 00: Error during parsing. Encountered " <PATH> "\\ "" at line 2, column
Was expecting one of:
<EOF>
"cat" ...
"clear" ...
"fs" ...
"sh" ...
"cd" ...
"cp" ...
"copyFromLocal" ...
"copyToLocal" ...
"dump" ...
"\\d" ...
"describe" ...
"\\de" ...
"aliases" ...
"explain" ...
"\\e" ...
"help" ...
"history" ...
"kill" ...
"ls" ...
"mv" ...
"mkdir" ...
"pwd" ...
"quit" ...
"\\q" ...
"register" ...
"rm" ...
"rmf" ...
"set" ...
"illustrate" ...
"\\i" ...
"run" ...
"exec" ...
"%default" ...
"%declare" ...
"scriptDone" ...
"" ...
"" ...
<EOL> ...
ERROR 12
ERROR 12
ERROR 12
ERROR 12
ERROR 10
1.
";" ...
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt> largest1 = FILTER count1 by $1>2;
ncountered Warning IMPLICIT_CAST_TO_LONG 1 time(s).
grunt> DUMP largest1
ncountered Warning IMPLICIT_CAST_TO_LONG 1 time(s).
2016-03-19 07:22:47,335 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: GROUP_BY,FILTER
se
ne.util.CombinerOptimizerUtil - Choosing to move algebraic foreach to combiner
initialized

quired reducers.
.
ready initialized
#setJar(String).
es
.address

cksum
00_0
version is 1
Total Length = 3386
Input split[0]:
Length = 3386
Locations:
being processed file:/root/essay:0+3386
ate code.
[-1,-1],count1[8,9],group1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9],larges
t1[9,11]

); length = 2373/6553600
000_0
fle@4f578be8
tsThreshold=10
022349_0007_m_000000_0
ed.. Returning
on-disk map-outputs
uce memory limit

job phase (AliasName[line,offset]): M: input1[4,9],words1[-1,-1],count1[8,9],gro
up1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9],largest1[9,11]
committing
_r_000000_0' to file:/tmp/temp2073304142/tmp-479985687/_temporary/0/task_local17
26022349_0007_r_000000
ne.mapReduceLayer.MapReduceLauncher - Processing aliases count1,group1,input1,la
rgest1,words1
-1,-1],count1[8,9],group1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9],largest
1[9,11]
]
initialized
initialized
initialized

UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 07:22:47
2016-03-19 07:22:48
,FILTER
Features
GROUP_BY
Success!
JobId Maps
Reduces MaxMapTime
MinMapTime
AvgMapTime
MedianMa
ature Outputs
job_local1726022349_0007
1
1
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
count1,group1,input1,largest1,words1 GROUP_BY,C
OMBINER
file:/tmp/temp2073304142/tmp-479985687,
Input(s):
Output(s):
Successfully stored 41 records in: "file:/tmp/temp2073304142/tmp-479985687"
Counters:
Job DAG:
job_local1726022349_0007
initialized
initialized
initialized
(a,11)
(an,3)
(as,4)
(in,12)
(is,9)
(of,15)
(on,3)
(or,5)
(to,13)
(us,3)
(The,6)
(and,9)
(any,4)
(are,7)
(buy,4)
(for,6)
(not,5)
(one,3)
(our,3)
(the,27)
(you,13)
(Just,4)
(What,3)
(just,3)
(need,5)
(that,6)
(with,5)
(your,7)
(phone,10)
(Mobile,6)
(Phones,4)
(buying,3)
(mobile,26)
(person,3)
(phone.,3)
(phones,14)
(social,3)
(don t,3)
(message,3)
(Internet,3)
(important,5)
grunt>
grunt>
grunt>
grunt>
grunt> quit
2016-03-19 07:24:18,254 [main] INFO org.apache.pig.Main - Pig script completed
in 33 minutes, 43 seconds and 327 milliseconds (2023327 ms)
root@ip-172-31-37-176:~# bin dev home
lib
lost+found mnt proc ru
n srv tmp var
No command 'bin' found, did you mean:
Command 'bing' from package 'bing' (universe)
Command 'win' from package 'wily' (universe)
Command 'ben' from package 'ben' (universe)
Command 'bins' from package 'bins' (universe)
Command 'bip' from package 'bip' (universe)
Command 'tin' from package 'tin' (universe)
Command 'bio' from package 'barry-util' (universe)
Command 'bino' from package 'bino' (universe)
Command 'din' from package 'din' (universe)
bin: command not found
root@ip-172-31-37-176:~# boot etc initrd.img lib64 media
opt root sb
in sys usr vmlinuz
No command 'boot' found, did you mean:
Command 'boog' from package 'alliance' (universe)
Command 'booth' from package 'booth' (universe)

Command 'brot' from package 'fortunes-de' (universe)
Command 'boom' from package 'prboom-plus' (universe)
Command 'boom' from package 'alliance' (universe)
Command 'hboot' from package 'lam-runtime' (universe)
Command 'booi' from package 'boo' (universe)
Command 'root' from package 'root-system-bin' (universe)
Command 'booc' from package 'boo' (universe)
boot: command not found
root@ip-172-31-37-176:~# bin dev home
lib
lost+found mnt proc ru
n srv tmp var
No command 'bin' found, did you mean:
Command 'win' from package 'wily' (universe)
Command 'ben' from package 'ben' (universe)
Command 'bing' from package 'bing' (universe)
Command 'bip' from package 'bip' (universe)
Command 'bino' from package 'bino' (universe)
Command 'din' from package 'din' (universe)
Command 'tin' from package 'tin' (universe)
Command 'bio' from package 'barry-util' (universe)
Command 'bins' from package 'bins' (universe)
bin: command not found
root@ip-172-31-37-176:~# boot etc initrd.img lib64 media
opt root sb
in sys usr vmlinuz
No command 'boot' found, did you mean:
Command 'boog' from package 'alliance' (universe)
Command 'root' from package 'root-system-bin' (universe)
Command 'booth' from package 'booth' (universe)
Command 'booc' from package 'boo' (universe)
Command 'brot' from package 'fortunes-de' (universe)
Command 'booi' from package 'boo' (universe)
Command 'hboot' from package 'lam-runtime' (universe)
Command 'boom' from package 'prboom-plus' (universe)
Command 'boom' from package 'alliance' (universe)
boot: command not found
root@ip-172-31-37-176:~#
login as: ubuntu

Authenticating with public key "imported-openssh-key"
Welcome to Ubuntu 14.04.3 LTS (GNU/Linux 3.13.0-74-generic x86_64)
* Documentation: https://help.ubuntu.com/
System information as of Sat Mar 19 04:15:18 UTC 2016
System load:
Usage of /:
Memory usage:
Swap usage:
0.27
24.5% of 7.74GB
6%
0%
Processes:
102
Users logged in:
0
IP address for eth0: 172.31.37.176
Graph this data and manage this system at:

https://landscape.canonical.com/
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
67 packages can be updated.

37 updates are security updates.
Last login: Sat Mar 19 04:15:18 2016 from 182.73.111.182
ubuntu@ip-172-31-37-176:~$ sudo -i
root@ip-172-31-37-176:~# pig -x local
16/03/19 06:31:01 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
16/03/19 06:31:01 INFO pig.ExecTypeProvider: Picked LOCAL as the ExecType
2016-03-19 06:31:01,738 [main] INFO org.apache.pig.Main - Apache Pig version 0.
15.0 (r1682971) compiled J
un 01 2015, 11:44:35
2016-03-19 06:31:01,738 [main] INFO org.apache.pig.Main - Logging error message
s to: /root/pig_1458369061
737.log
2016-03-19 06:31:01,757 [main] INFO org.apache.pig.impl.util.Utils - Default bo
otup file /root/.pigbootup
not found
ation - fs.default.name is
deprecated. Instead, use fs.defaultFS
ation - mapred.job.tracker
is deprecated. Instead, use mapreduce.jobtracker.addr
ess
ne.HExecutionEngine - Conn
ecting to hadoop file system at: file:///
ation - io.bytes.per.check
sum is deprecated. Instead, use dfs.bytes-per-checksum
grunt> Empemail = LOAD '/root/input/EmpEmailID.txt' as (id:int , email:chararray
);
ation - fs.default.name is
deprecated. Instead, use fs.defaultFS
ation - io.bytes.per.check
sum is deprecated. Instead, use dfs.bytes-per-checksum
grunt> dump Empemail
2016-03-19 06:32:33,746 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: UNKNOWN
se

ation - session.id is deprecated. Instead, use dfs.metrics.session-id
2016-03-19 06:32:34,103 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Initializing JVM Metrics with processName=JobTracker, sessionId=
ation - mapred.job.reduce.markreset.buffer.percent is deprecated. Instead, use m
apreduce.reduce.markreset.buffer.percent
ation - mapred.output.compress is deprecated. Instead, use mapreduce.output.file
outputformat.compress
.
ation - mapred.job.tracker.http.address is deprecated. Instead, use mapreduce.jo
btracker.http.address
ready initialized
#setJar(String).
ter - Cleaning up the staging area file:/tmp/hadoop-root/mapred/staging/root9357
16635/.staging/job_local935716635_0001
2016-03-19 06:32:34,572 [JobControl] INFO org.apache.hadoop.mapreduce.lib.jobco
ntrol.ControlledJob - PigLatin:DefaultJobName got an error while submitting
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path doe
s not exist: file:/root/input/EmpEmailID.txt
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInput
Format.getSplits(PigInputFormat.java:279)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.
java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.jav
a:318)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitt
er.java:196)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma
tion.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(Contr
olledJob.java:335)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.ja
va:128)
at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:
194)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduc
eLauncher$1.run(MapReduceLauncher.java:276)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input pa
th does not exist: file:/root/input/EmpEmailID.txt
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedL
istStatus(FileInputFormat.java:323)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(File
InputFormat.java:265)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextI
nputFormat.listStatus(PigTextInputFormat.java:36)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileI
nputFormat.java:387)
... 18 more
ne.mapReduceLayer.MapReduceLauncher - Processing aliases Empemail
ne.mapReduceLayer.MapReduceLauncher - detailed locations: M: Empemail[1,11],Empe
mail[-1,-1] C: R:
2016-03-19 06:32:34,796 [main] WARN org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_
on_failure if you want Pig to stop immediately on failure.
ne.mapReduceLayer.MapReduceLauncher - job job_local935716635_0001 has failed! St
op running all dependent jobs
initialized
2016-03-19 06:32:34,803 [main] ERROR org.apache.pig.tools.pigstats.PigStats - ER
ROR 0: java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING
2016-03-19 06:32:34,803 [main] ERROR org.apache.pig.tools.pigstats.mapreduce.MRP
igStatsUtil - 1 map reduce job(s) failed!
UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 06:32:34
2016-03-19 06:32:34
Features
UNKNOWN
Failed!
Failed Jobs:
JobId Alias Feature Message Outputs
job_local935716635_0001 Empemail
MAP_ONLY
Message: org.apache.pig.
backend.executionengine.ExecException: ERROR 2118: Input path does not exist: fi
le:/root/input/EmpEmailID.txt
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.
java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.jav
a:318)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitt
er.java:196)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma
tion.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(Contr
olledJob.java:335)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.ja
va:128)
at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:
194)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduc
eLauncher$1.run(MapReduceLauncher.java:276)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input pa
th does not exist: file:/root/input/EmpEmailID.txt
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedL
istStatus(FileInputFormat.java:323)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(File
InputFormat.java:265)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextI
nputFormat.listStatus(PigTextInputFormat.java:36)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileI
nputFormat.java:387)
... 18 more
file:/tmp/temp666787139/tmp-2036215924,
Input(s):
Failed to read data from "/root/input/EmpEmailID.txt"
Output(s):
Failed to produce result in "file:/tmp/temp666787139/tmp-2036215924"
Counters:
Job DAG:
job_local935716635_0001
ne.mapReduceLayer.MapReduceLauncher - Failed!
66: Unable to open iterator for alias Empemail. Backend error : java.lang.Illega
lStateException: Job in state DEFINE instead of RUNNING
grunt> Empemail = LOAD '/root/input/EmpEmailId.txt' as (id:int , email:chararray
);
grunt> dump Empemail
2016-03-19 06:33:56,264 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: UNKNOWN
se
initialized

.
ready initialized
#setJar(String).
ne.mapReduceLayer.MapReduceLauncher - Processing aliases Empemail
mail[-1,-1] C: R:
]
2016-03-19 06:33:57,366 [Thread-6] INFO org.apache.hadoop.mapred.LocalJobRunner
- OutputCommitter set in config null
2016-03-19 06:33:57,412 [Thread-6] INFO org.apache.hadoop.conf.Configuration.de
precation - fs.default.name is deprecated. Instead, use fs.defaultFS
precation - mapred.job.reduce.markreset.buffer.percent is deprecated. Instead, u
se mapreduce.reduce.markreset.buffer.percent
precation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.
address
precation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-chec
ksum
2016-03-19 06:33:57,413 [Thread-6] INFO org.apache.hadoop.mapreduce.lib.output.
FileOutputCommitter - File Output Committer Algorithm version is 1
- OutputCommitter is org.apache.pig.backend.hadoop.executionengine.mapReduceLay
er.PigOutputCommitter
- Waiting for map tasks

00_0
version is 1
Total Length = 237
Input split[0]:
Length = 237
Locations:
being processed file:/root/input/EmpEmailId.txt:0+237
version is 1
ate code.
ig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map - Aliases being
processed per job phase (AliasName[line,offset]): M: Empemail[2,11],Empemail[-1,
-1] C: R:
adoop.mapred.Task - Task attempt_local1224478176_0002_m_000000_0 is allowed to c
ommit now
adoop.mapreduce.lib.output.FileOutputCommitter - Saved output of task 'attempt_l
ocal1224478176_0002_m_000000_0' to file:/tmp/temp666787139/tmp1022024822/_tempor
ary/0/task_local1224478176_0002_m_000000
000_0
- map task executor complete.
initialized
initialized
ation - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps

ation - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
initialized
UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 06:33:56
2016-03-19 06:33:57
Features
UNKNOWN
Success!
JobId Maps
Reduces MaxMapTime
MinMapTime
AvgMapTime
MedianMa
ature Outputs
job_local1224478176_0002
1
0
n/a
n/a
n/a
n/a
0
0
0
0
Empemail
MAP_ONLY
file:/tmp/temp6667
87139/tmp1022024822,
Input(s):
Successfully read 11 records from: "/root/input/EmpEmailId.txt"
Output(s):
Counters:
Job DAG:
job_local1224478176_0002
initialized
initialized
initialized

(1,Abraham@company.com)
(2,Jacob@company.com)
(3,John@company.com)
(4,Marley@company.com)
(5,Bob@company.com)
(6,Wincent@company.com)
(7,Steve@company.com)
(8,Michael@company.com)
(9,Stewart@company.com)
(10,Daniel@company.com)
(11,Jack@company.com)
grunt> joined = JOIN Emprec by $0 , Empemail $0;
00: <line 3, column 38> mismatched input '$0' expecting BY
grunt> joined = JOIN Emprec by $0 , Empemail by $0;
00: Pig script failed to parse:
<line 3, column 14> Undefined alias: Emprec
00: Pig script failed to parse:
<line 3, column 14> Undefined alias: Emprec
grunt> Emprec = LOAD '/root/input/EmpRecord.txt' AS (id:int , name:chararray , s
alary:int , location:chararray , DOJ:chararray);
grunt> dunmp joined
00: Error during parsing. Encountered " <IDENTIFIER> "dunmp "" at line 3, column
1.
Was expecting one of:
<EOF>
"cat" ...
"clear" ...
"fs" ...
"sh" ...
"cd" ...
"cp" ...
"copyFromLocal" ...
"copyToLocal" ...
"dump" ...
"\\d" ...
"describe" ...
"\\de" ...
"aliases" ...
"explain" ...
"\\e" ...
"help" ...
"history" ...
"kill" ...
"ls" ...
"mv" ...
"mkdir" ...
"pwd" ...
"quit" ...
"\\q" ...
"register" ...
"rm" ...
"rmf" ...
"set" ...
"illustrate" ...
"\\i" ...
"run" ...
"exec" ...
"%default" ...
"%declare" ...
"scriptDone" ...
"" ...
"" ...
<EOL> ...
";" ...
grunt> dump filtered
03: Unable to find an operator for alias filtered
grunt> dump joined
2016-03-19 06:43:10,258 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: HASH_JOIN
se
ne.mapReduceLayer.MRCompiler$LastInputStreamingOptimizer - Rewrite: POPackage->P
OForEach to POPackage(JoinPackager)
initialized

quired reducers.
.
ready initialized
#setJar(String).
es
.address
cksum
_0
version is 1
Total Length = 400
Input split[0]:
Length = 400
Locations:
being processed file:/root/input/EmpRecord.txt:0+400
ate code.
ne.mapReduceLayer.MapReduceLauncher - Processing aliases Empemail,Emprec,joined
ne.mapReduceLayer.MapReduceLauncher - detailed locations: M: Emprec[3,9],Emprec[
-1,-1],joined[4,9],Empemail[2,11],Empemail[-1,-1],joined[4,9] C: R:
es being processed per job phase (AliasName[line,offset]): M: Emprec[3,9],Emprec
[-1,-1],joined[4,9],Empemail[2,11],Empemail[-1,-1],joined[4,9] C: R:
); length = 41/6553600
adoop.mapred.Task - Task:attempt_local16493891_0003_m_000000_0 is done. And is i
n the process of committing
0_0
_0
version is 1
Total Length = 237
Input split[0]:
Length = 237
Locations:
2016-03-19 06:43:11,078 [LocalJobRunner Map Task Executor #0] WARN org.apache.p

ig.data.SchemaTupleBackend - SchemaTupleBackend has already been initialized
es being processed per job phase (AliasName[line,offset]): M: Emprec[3,9],Emprec
[-1,-1],joined[4,9],Empemail[2,11],Empemail[-1,-1],joined[4,9] C: R:
); length = 41/6553600
adoop.mapred.Task - Task:attempt_local16493891_0003_m_000001_0 is done. And is i
n the process of committing
1_0
2016-03-19 06:43:11,098 [pool-5-thread-1] INFO org.apache.hadoop.mapred.LocalJo
bRunner - Starting task: attempt_local16493891_0003_r_000000_0
2016-03-19 06:43:11,110 [pool-5-thread-1] INFO org.apache.hadoop.mapreduce.lib.
output.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 06:43:11,120 [pool-5-thread-1] INFO org.apache.hadoop.mapred.ReduceT
ask - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuff
le@2953cd26
2016-03-19 06:43:11,139 [pool-5-thread-1] INFO org.apache.hadoop.mapreduce.task
.reduce.MergeManagerImpl - MergerManager: memoryLimit=709551680, maxSingleShuffl
eLimit=177387920, mergeThreshold=468304128, ioSortFactor=10, memToMemMergeOutput
sThreshold=10
891_0003_m_000000_0
reduce.MergeManagerImpl - closeInMemoryFile -> map-output of size: 538, inMemory
MapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->538

891_0003_m_000001_0
ed.. Returning
bRunner - 2 / 2 copied.
.reduce.MergeManagerImpl - finalMerge called with 2 in-memory map-outputs and 0
on-disk map-outputs
hod)
ativeIO.java:267)
Pool.java:206)
java:1145)
.java:615)
.reduce.MergeManagerImpl - Merged 2 segments, 887 bytes to disk to satisfy reduc
e memory limit
.reduce.MergeManagerImpl - Merging 1 files, 889 bytes from disk
.reduce.MergeManagerImpl - Merging 0 segments, 0 bytes from memory into reduce
output.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 06:43:11,324 [pool-5-thread-1] INFO org.apache.hadoop.conf.Configura
tion.deprecation - mapred.skip.on is deprecated. Instead, use mapreduce.job.skip
records
2016-03-19 06:43:11,341 [pool-5-thread-1] WARN org.apache.pig.data.SchemaTupleB
ackend - SchemaTupleBackend has already been initialized
2016-03-19 06:43:11,356 [pool-5-thread-1] INFO org.apache.pig.backend.hadoop.ex
ecutionengine.mapReduceLayer.PigMapReduce$Reduce - Aliases being processed per j
ob phase (AliasName[line,offset]): M: Emprec[3,9],Emprec[-1,-1],joined[4,9],Empe
mail[2,11],Empemail[-1,-1],joined[4,9] C: R:
2016-03-19 06:43:11,386 [pool-5-thread-1] INFO org.apache.hadoop.mapred.Task -
Task:attempt_local16493891_0003_r_000000_0 is done. And is in the process of com

mitting
output.FileOutputCommitter - Saved output of task 'attempt_local16493891_0003_r_
000000_0' to file:/tmp/temp666787139/tmp568016623/_temporary/0/task_local1649389
1_0003_r_000000
bRunner - reduce > reduce
bRunner - Finishing task: attempt_local16493891_0003_r_000000_0
initialized
initialized
initialized
UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 06:43:10
2016-03-19 06:43:11
N
Features
HASH_JOI
Success!
JobId Maps
Reduces MaxMapTime
MinMapTime
AvgMapTime
MedianMa
ature Outputs
job_local16493891_0003 2
1
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
Empemail,Emprec,joined HASH_JOIN
file:/tmp/temp6667
87139/tmp568016623,
Input(s):
Successfully read 11 records from: "/root/input/EmpRecord.txt"
Output(s):
Counters:

Job DAG:
job_local16493891_0003
initialized
initialized
initialized
(1,Abraham,11054,New York,10-Jan-20012,1,Abraham@company.com)
(2,Jacob,21541,New York,16-Feb-2011,2,Jacob@company.com)
(3,John,32541,Washington,22-Mar-2010,3,John@company.com)
(4,Marley,43658,Detroit,28-Apr-2009,4,Marley@company.com)
(5,Bob,57854,Michigan,4-May-2008,5,Bob@company.com)
(6,Wincent,65241,California,9-Jun-2007,6,Wincent@company.com)
(7,Steve,79854,California,15-Jul-2006,7,Steve@company.com)
(8,Michael,85412,Mexico,21-Aug-2006,8,Michael@company.com)
(9,Stewart,92789,Brazil,29-Sep-2005,9,Stewart@company.com)
(10,Daniel,102545,New York,5-Oct-2004,10,Daniel@company.com)
(11,Jack,0,Washington,11-Nov-2003,11,Jack@company.com)
grunt> joined1 = COGROUP Emprec by $0 , Empemail by $0;
grunt> dump joined1
2016-03-19 06:44:12,763 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: COGROUP
se

initialized
quired reducers.
.
ready initialized
#setJar(String).

es
.address
cksum
00_0
version is 1
Total Length = 400
Input split[0]:
Length = 400
Locations:
being processed file:/root/input/EmpRecord.txt:0+400
ate code.
es being processed per job phase (AliasName[line,offset]): M: Empemail[2,11],Emp
email[-1,-1],joined1[5,10],Emprec[3,9],Emprec[-1,-1],joined1[5,10] C: R:
); length = 41/6553600
000_0
01_0
version is 1
Total Length = 237
Input split[0]:
Length = 237
Locations:
2016-03-19 06:44:13,238 [LocalJobRunner Map Task Executor #0] WARN org.apache.p

ig.data.SchemaTupleBackend - SchemaTupleBackend has already been initialized
); length = 41/6553600
001_0
fle@2828e2c5
tsThreshold=10
81714_0004_m_000001_0

81714_0004_m_000000_0
ed.. Returning
on-disk map-outputs
k.reduce.MergeManagerImpl - Merged 2 segments, 887 bytes to disk to satisfy redu
ce memory limit
hod)
ativeIO.java:267)
Pool.java:206)
java:1145)
.java:615)
job phase (AliasName[line,offset]): M: Empemail[2,11],Empemail[-1,-1],joined1[5,
10],Emprec[3,9],Emprec[-1,-1],joined1[5,10] C: R:
committing
_r_000000_0' to file:/tmp/temp666787139/tmp-278931343/_temporary/0/task_local123
6981714_0004_r_000000
ne.mapReduceLayer.MapReduceLauncher - Processing aliases Empemail,Emprec,joined1
mail[-1,-1],joined1[5,10],Emprec[3,9],Emprec[-1,-1],joined1[5,10] C: R:
]
initialized
initialized
initialized
UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 06:44:12
2016-03-19 06:44:13
Features
COGROUP
Success!
JobId Maps
Reduces MaxMapTime
MinMapTime
AvgMapTime
MedianMa
ature Outputs
job_local1236981714_0004
2
1
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
Empemail,Emprec,joined1
COGROUP file:/tmp/
temp666787139/tmp-278931343,
Input(s):
Successfully read 11 records from: "/root/input/EmpRecord.txt"
Output(s):
Successfully stored 11 records in: "file:/tmp/temp666787139/tmp-278931343"
Counters:

Job DAG:
job_local1236981714_0004
initialized
initialized
initialized
(1,{(1,Abraham,11054,New York,10-Jan-20012)},{(1,Abraham@company.com)})
(2,{(2,Jacob,21541,New York,16-Feb-2011)},{(2,Jacob@company.com)})
(3,{(3,John,32541,Washington,22-Mar-2010)},{(3,John@company.com)})
(4,{(4,Marley,43658,Detroit,28-Apr-2009)},{(4,Marley@company.com)})
(5,{(5,Bob,57854,Michigan,4-May-2008)},{(5,Bob@company.com)})
(6,{(6,Wincent,65241,California,9-Jun-2007)},{(6,Wincent@company.com)})
(7,{(7,Steve,79854,California,15-Jul-2006)},{(7,Steve@company.com)})
(8,{(8,Michael,85412,Mexico,21-Aug-2006)},{(8,Michael@company.com)})
(9,{(9,Stewart,92789,Brazil,29-Sep-2005)},{(9,Stewart@company.com)})
(10,{(10,Daniel,102545,New York,5-Oct-2004)},{(10,Daniel@company.com)})
(11,{(11,Jack,0,Washington,11-Nov-2003)},{(11,Jack@company.com)})
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt> dump joined1 $1.$1;
2016-03-19 06:49:20,662 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: COGROUP

se
initialized
quired reducers.
.
ready initialized
#setJar(String).

es
.address
cksum
00_0
version is 1
Total Length = 400
Input split[0]:
Length = 400
Locations:
----------------------2016-03-19 06:49:20,972 [LocalJobRunner Map Task Executor #0] INFO
ig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader being processed file:/root/input/EmpRecord.txt:0+400
org.apache.p
Current split
org.apache.h
org.apache.h
org.apache.h
org.apache.h

ate code.
); length = 41/6553600
000_0
01_0
version is 1
Total Length = 237
Input split[0]:
Length = 237
Locations:
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000dd8fa000, 10
4861696, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.

# Native memory allocation (malloc) failed to allocate 104861696 bytes for commi
tting reserved memory.
# An error report file with more information is saved as:
# /root/hs_err_pid3506.log
root@ip-172-31-37-176:~#

Hadoop Day4

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Hadoop Day4

Uploaded by

Copyright:

Available Formats

(foremost)

adoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100

2016-03-19 07:03:08,231 [localfetcher#2] INFO org.apache.hadoop.mapreduce.task.

2016-03-19 07:05:39,921 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - E

ics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - al

being processed file:/root/essay:0+3386

2016-03-19 07:05:40,516 [pool-17-thread-1] WARN org.apache.pig.data.SchemaTuple

grunt> largest1 = filter count1 as $1>2;

2016-03-19 07:22:47,397 [main] INFO org.apache.pig.tools.pigstats.mapreduce.MRS

eprecation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-che

adoop.mapred.MapTask - kvstart = 26214396(104857584); kvend = 26212024(104848096

2016-03-19 07:22:47,783 [pool-20-thread-1] INFO org.apache.hadoop.mapreduce.tas

2016-03-19 07:22:48,124 [main] INFO org.apache.pig.tools.pigstats.mapreduce.Sim

Command 'booth' from package 'booth' (universe)

login as: ubuntu

Graph this data and manage this system at:

67 packages can be updated.

ne.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1

ey [pig.schematuple] is false, will not generate code.

adoop.mapred.LocalJobRunner - Starting task: attempt_local1224478176_0002_m_0000

ation - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps

2016-03-19 06:33:57,937 [main] INFO org.apache.pig.backend.hadoop.executionengi

t is not set, set to default 0.3

2016-03-19 06:43:11,078 [LocalJobRunner Map Task Executor #0] WARN org.apache.p

l16493891_0003_m_000001_0 decomp: 349 len: 353 to MEMORY

Task:attempt_local16493891_0003_r_000000_0 is done. And is in the process of com

Total records proactively spilled: 0

2016-03-19 06:44:12,801 [main] INFO org.apache.hadoop.conf.Configuration.deprec

r - OutputCommitter set in config null

2016-03-19 06:44:13,238 [LocalJobRunner Map Task Executor #0] WARN org.apache.p

l1236981714_0004_m_000000_0 decomp: 538 len: 542 to MEMORY

Total records written : 11

ulator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeF

onengine.util.MapRedUtil - Total input paths to process : 1

adoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600

# There is insufficient memory for the Java Runtime Environment to continue.

You might also like