HDFS命令行操作
来源:互联网 发布:aes算法的加密 编辑:程序博客网 时间:2024/06/10 00:15
启动后可通过命令行使用hadoop。
(1)所有命令 (先将$HADOOP_HOME/bin加入到.bashrc的$PATH变量中)
[hadoop@node14 hadoop-0.21.0]$ ll $HADOOP_HOME/bintotal 88-rwxr-xr-x 1 hadoop hadoop 4131 Aug 17 2010 hadoop-rwxr-xr-x 1 hadoop hadoop 8658 Aug 17 2010 hadoop-config.sh-rwxr-xr-x 1 hadoop hadoop 3841 Aug 17 2010 hadoop-daemon.sh-rwxr-xr-x 1 hadoop hadoop 1242 Aug 17 2010 hadoop-daemons.sh-rwxr-xr-x 1 hadoop hadoop 4130 Aug 17 2010 hdfs-rwxr-xr-x 1 hadoop hadoop 1201 Aug 17 2010 hdfs-config.sh-rwxr-xr-x 1 hadoop hadoop 3387 Aug 17 2010 mapred-rwxr-xr-x 1 hadoop hadoop 1207 Aug 17 2010 mapred-config.sh-rwxr-xr-x 1 hadoop hadoop 2720 Aug 17 2010 rcc-rwxr-xr-x 1 hadoop hadoop 2058 Aug 17 2010 slaves.sh-rwxr-xr-x 1 hadoop hadoop 1367 Aug 17 2010 start-all.sh-rwxr-xr-x 1 hadoop hadoop 1018 Aug 17 2010 start-balancer.sh-rwxr-xr-x 1 hadoop hadoop 1778 Aug 17 2010 start-dfs.sh-rwxr-xr-x 1 hadoop hadoop 1255 Aug 17 2010 start-mapred.sh-rwxr-xr-x 1 hadoop hadoop 1359 Aug 17 2010 stop-all.sh-rwxr-xr-x 1 hadoop hadoop 1069 Aug 17 2010 stop-balancer.sh-rwxr-xr-x 1 hadoop hadoop 1277 Aug 17 2010 stop-dfs.sh-rwxr-xr-x 1 hadoop hadoop 1163 Aug 17 2010 stop-mapred.sh
(2) hadoop命令
[hadoop@node14 hadoop-0.21.0]$ hadoopUsage: hadoop [--config confdir] COMMAND where COMMAND is one of: fs run a generic filesystem user client version print the version jar <jar> run a jar file distcp <srcurl> <desturl> copy file or directories recursively archive -archiveName NAME -p <parent path> <src>* <dest> create a hadoop archive classpath prints the class path needed to get the Hadoop jar and the required libraries daemonlog get/set the log level for each daemon or CLASSNAME run the class named CLASSNAMEMost commands print help when invoked w/o parameters.
(3) hadoop fs
[hadoop@node14 hadoop-0.21.0]$ hadoop fsUsage: java FsShell [-ls <path>] [-lsr <path>] [-df [<path>]] [-du [-s] [-h] <path>] [-dus <path>] [-count[-q] <path>] [-mv <src> <dst>] [-cp <src> <dst>] [-rm [-skipTrash] <path>] [-rmr [-skipTrash] <path>] [-expunge] [-put <localsrc> ... <dst>] [-copyFromLocal <localsrc> ... <dst>] [-moveFromLocal <localsrc> ... <dst>] [-get [-ignoreCrc] [-crc] <src> <localdst>] [-getmerge <src> <localdst> [addnl]] [-cat <src>] [-text <src>] [-copyToLocal [-ignoreCrc] [-crc] <src> <localdst>] [-moveToLocal [-crc] <src> <localdst>] [-mkdir <path>] [-setrep [-R] [-w] <rep> <path/file>] [-touchz <path>] [-test -[ezd] <path>] [-stat [format] <path>] [-tail [-f] <file>] [-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...] [-chown [-R] [OWNER][:[GROUP]] PATH...] [-chgrp [-R] GROUP PATH...] [-help [cmd]]Generic options supported are-conf <configuration file> specify an application configuration file-D <property=value> use value for given property-fs <local|namenode:port> specify a namenode-jt <local|jobtracker:port> specify a job tracker-files <comma separated list of files> specify comma separated files to be co pied to the map reduce cluster-libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.-archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines.The general command line syntax isbin/hadoop command [genericOptions] [commandOptions]
(4)HDFS操作
hadoop fshadoop fs -lshadoop fs -mkdir firstdir//HDFS创建目录hadoop fs -rmr firstdir//HDFS删除目录hadoop fs -put test.txt first.txt//从本地目录中将文件放入HDFShadoop fs -cat first.txthadoop fs -dfhadoop fs -get first.txt FirstTXTfromHDFS.txt //从HDFS取文件到本地
若文件写入遇到异常
(0)检查和机器名是否正确
node14配置了外部IP和内部IP,在/etc/hosts中加入两条IP与机器名的对应表,如果外部IP放在内部IP的前面,
则通过netstat -npl查看时,发现9000和9001是外部IP占据,故应在/etc/hosts中内部IP放在外部的IP的前面。
或者在conf中配置文件中,全部用IP,而不要用机器名。
(1)关闭防火墙
sudo /etc/init.d/iptables stop
(2)查看磁盘空间是否正常
df -hl
(3)检查目录是否正常
hadoop.tmp.dir默认:/tmp/hadoop-${user.name}
删除/tmp下面的文件,重新hadoop namenode -format,重启所有进程。
(4)单独启动各个进程
在namenode和datanode上分别启动节点
$hadoop-daemon.sh start namenode
$hadoop-daemon.sh start datanode
- HDFS命令行操作
- HDFS命令行操作
- HDFS命令行操作
- HDFS命令行文件操作
- hdfs 命令行操作
- HDFS命令行操作演示
- HDFS命令行操作
- HDFS操作-命令行接口
- HDFS命令行文件操作
- hdfs命令行操作
- HDFS的相关命令行操作
- HDFS的命令行操作及可视化
- HDFS的shell(命令行客户端)操作
- hadoop2.4.1伪分布式安装,hdfs命令行操作学习笔记
- HDFS java 操作 HDFS
- HDFS命令行接口
- HDFS中的命令行
- HDFS shell命令行详解
- 38条PHP优化加速方案
- [iOS学习笔记] 如何在Xcode 4中修改代码的Copyright信息
- jsp ajax实例讲解
- js中的继承实现
- tel.xls.exe U盘病毒的查杀方法总结
- HDFS命令行操作
- CDatabase::ExecuteSQL()
- 监听控件值的改变
- 进程与线程的区别
- Spring StoredProcedure for Oracle cursor
- iPhone开发--UIActionSheet的按钮不灵敏
- 格式化输出
- poj 1436 Horizontally Visible Segments(线段树 区间的覆盖关系)
- Nginx目录设置访问验证