python - Hadoop Streaming - Unable to find file error -


i trying run hadoop-streaming python job. 

bin/hadoop jar contrib/streaming/hadoop-0.20.1-streaming.jar  -d stream.non.zero.exit.is.failure=true  -input /ixml  -output /oxml  -mapper scripts/mapper.py  -file scripts/mapper.py  -inputreader "streamxmlrecordreader,begin=channel,end=/channel"  -jobconf mapred.reduce.tasks=0

i made sure mapper.py has permissions. errors out saying

caused by: java.io.ioexception: cannot run program "mapper.py":      error=2, no such file or directory     @ java.lang.processbuilder.start(processbuilder.java:460)     @ org.apache.hadoop.streaming.pipemapred.configure(pipemapred.java:214) ... 19 more caused by: java.io.ioexception: error=2, no such file or directory     @ java.lang.unixprocess.forkandexec(native method)     @ java.lang.unixprocess.(unixprocess.java:53)     @ java.lang.processimpl.start(processimpl.java:91)     @ java.lang.processbuilder.start(processbuilder.java:453)

i tried copying mapper.py hdfs , give same hdfs://localhost/mapper.py link, not work too! thoughts on how fix bug?.

looking @ example on hadoopstreaming wiki page, seems should change

-mapper scripts/mapper.py  -file scripts/mapper.py

to

-mapper mapper.py  -file scripts/mapper.py

since "shipped files go working directory". might need specify python interpreter directly:

-mapper /path/to/python mapper.py  -file scripts/mapper.py

Comments

Post a Comment

Popular posts from this blog

Add email recipient to all new Trac tickets -

is there configuration change can made trac send notification email address upon creation of new tickets? if can't done through config, plugin second best option, source code modification last resort. note: setting smtp_always_cc in notification section of tracini send messages on updates. i'm in need of email notifications only on creation of new ticket. does have email notification? 1 option have create query list ten (or many) recently-created tickets. when looking @ results of query, use "rss feed" button @ bottom of page subscribe feed notify whenever results of query updated.
Read more

400 Bad Request on Apache/PHP AddHandler wrapper -

i'm trying create wrapper/handler called on apache server whenever requests php script inside of directory. way can authorize users entire directory or write other stuff called when directory called. this best configuration i've been able come with... <directory "/srv/http/innov/public_html"> options -indexes allowoverride order allow,deny allow directoryindex index.php </directory> then in /srv/http/innov/public_html/kb/ have .htaccess file... options -indexes addhandler auth_handler .php action auth_handler ../auth_handler.php then in /srv/http/innov/public_html/kb/auth_handler.php follows... <?php $file = $_server['path_translated']; echo $file; ?> access log: - - [02/dec/2010:17:43:15 -0500] "get /kb/index.php http/1.1" 400 590 error log: [thu dec 02 17:50:19 2010] [error] [client xxx.xxx.xxx.xxx] invalid uri in request /kb/ http/1.1 i've checked browser , seems making proper request...
Read more

php - Change action and image src url's with jQuery -

i parsing form 1 of sites using php proxy. working fine form action , img src of submit button url's relative. how can prepend full http//www.domain.com/ in front of each of url's in form? edit: here's form code once fed through new domain: <div style="display: none;" id="subscribeform" class="newform"> <h5>subscribe our newsletter</h5> <form action="/campaignprocess.aspx?listid=27057" method="post" onsubmit="return checkwholeform93426(this)" name="catemaillistform93426"> <div class="form"> <div class="item"> <label for="clfullname">full name</label> <br /> <input type="text" maxlength="255" id="clfullname" name="fullname" class="cat_textbox_small" st...
Read more