摘要:原因没有安装源码参见调用链中函数找不到安装目录引起的报
$ ./nutch inject urls InjectorJob: starting at 2016-11-08 11:20:12 InjectorJob: Injecting urlDir: urls InjectorJob: Using class org.apache.gora.mongodb.store.MongoStore as the Gora st orage class. InjectorJob: java.lang.NullPointerException at java.lang.ProcessBuilder.start(ProcessBuilder.java:1010) at org.apache.hadoop.util.Shell.runCommand(Shell.java:482) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java: 702) at org.apache.hadoop.util.Shell.execCommand(Shell.java:791) at org.apache.hadoop.util.Shell.execCommand(Shell.java:774) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSys tem.java:646) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.jav a:434) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:28 1) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmi ssionFiles.java:125) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitt er.java:348) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma tion.java:1614) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303) at org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:115) at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:231) at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:252) at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:275) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:284)
原因没有安装hadoop.
源码参见:hadoop-common-{version}.jar , Shell.java line 370 : winUtilsPath = getQualifiedBinPath("winutils.exe"); 调用链中:函数checkHadoopHome() 找不到hadoop安装目录引起的报NPE
文章版权归作者所有,未经允许请勿转载,若此文章存在违规行为,您可以联系管理员删除。
转载请注明本文地址:https://www.ucloud.cn/yun/65273.html
摘要:在这之前,还是有必要对一些概念超轻量级反爬虫方案后端掘金前言爬虫和反爬虫日益成为每家公司的标配系统。 爬虫修炼之道——从网页中提取结构化数据并保存(以爬取糗百文本板块所有糗事为例) - 后端 - 掘金欢迎大家关注我的专题:爬虫修炼之道 上篇 爬虫修炼之道——编写一个爬取多页面的网络爬虫主要讲解了如何使用python编写一个可以下载多页面的爬虫,如何将相对URL转为绝对URL,如何限速,...
阅读 789·2023-04-26 00:37
阅读 671·2021-11-24 09:39
阅读 2099·2021-11-23 09:51
阅读 3657·2021-11-22 15:24
阅读 638·2021-10-19 11:46
阅读 1832·2019-08-30 13:53
阅读 2302·2019-08-29 17:28
阅读 1280·2019-08-29 14:11