Apache Pig Diagnostic运算符

由 jarodhu 创建，youj 最后一次修改 2016-12-28 Load 语句会简单地将数据加载到Apache Pig中的指定关系中。要验证Load语句的执行，必须使用Diagnostic运算符。Pig Latin提供四种不同类型的诊断运算符: Dump运算符Describe运算符Explanation运算符Illustration运算符在本章中，我们将讨论Pig Latin的Dump运算符。Dump运算符 Dump 运算符用于运行Pig Latin语句，并在屏幕上显示结果，它通常用于调试目的。语法下面给出了 Dump 运算符的语法。grunt> Dump Relation_Name例假设在HDFS中有一个包含以下内容的文件 student_data.txt 。001,Rajiv,Reddy,9848022337,Hyderabad002,siddarth,Battacharya,9848022338,Kolkata003,Rajesh,Khanna,9848022339,Delhi004,Preethi,Agarwal,9848022330,Pune005,Trupthi,Mohanthy,9848022336,Bhuwaneshwar006,Archana,Mishra,9848022335,Chennai.我们使用LOAD运算符将它读入关系 student ，如下所示。grunt> student = LOAD 'hdfs://localhost:9000/pig_data/student_data.txt' USING PigStorage(',') as ( id:int, firstname:chararray, lastname:chararray, phone:chararray, city:chararray );现在，使用Dump运算符打印关系的内容，如下所示。grunt> Dump student一旦执行上述 Pig Latin 语句，将启动一个MapReduce作业以从HDFS读取数据，将产生以下输出。2015-10-01 15:05:27,642 [main]INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete2015-10-01 15:05:27,652 [main]INFO org.apache.pig.tools.pigstats.mapreduce.SimplePigStats - Script Statistics: HadoopVersion PigVersion UserId StartedAt FinishedAt Features 2.6.0 0.15.0 Hadoop 2015-10-01 15:03:11 2015-10-01 05:27 UNKNOWN Success! Job Stats (time in seconds): JobId job_14459_0004Maps 1 Reduces 0 MaxMapTime n/a MinMapTime n/aAvgMapTime n/a MedianMapTime n/aMaxReduceTime 0MinReduceTime 0 AvgReduceTime 0MedianReducetime 0Alias student Feature MAP_ONLY Outputs hdfs://localhost:9000/tmp/temp580182027/tmp757878456,Input(s): Successfully read 0 records from: "hdfs://localhost:9000/pig_data/student_data.txt" Output(s): Successfully stored 0 records in: "hdfs://localhost:9000/tmp/temp580182027/tmp757878456" Counters: Total records written : 0 Total bytes written : 0 Spillable Memory Manager spill count : 0Total bags proactively spilled: 0 Total records proactively spilled: 0 Job DAG: job_1443519499159_0004 2015-10-01 15:06:28,403 [main]INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLau ncher - Success!2015-10-01 15:06:28,441 [main] INFO org.apache.pig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not generate code.2015-10-01 15:06:28,485 [main]INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 12015-10-01 15:06:28,485 [main]INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input pathsto process : 1(1,Rajiv,Reddy,9848022337,Hyderabad)(2,siddarth,Battacharya,9848022338,Kolkata)(3,Rajesh,Khanna,9848022339,Delhi)(4,Preethi,Agarwal,9848022330,Pune)(5,Trupthi,Mohanthy,9848022336,Bhuwaneshwar)(6,Archana,Mishra,9848022335,Chennai)