Bootstrap

Hadoop家族环境搭建各种版本选择 原创

System Requirements

Supported Platforms

•     GNU/Linuxis supported as a development and production platform for both server andclient.

•     SunSolaris is supported as a development and production platform for both serverand client.

•     FreeBSDis supported as a development and production platform for both server andclient.

•     Win32is supported as a development platformonly for both server and client.

•     Win64is supported as a development platformonly for both server and client.

•     MacOSXis supported as a development platformonly for both server and client.

 

Hive™ Requirements

•     Java1.7
Note:  Hive versions 1.2 onward require Java 1.7 or newer.Hive versions 0.14 to 1.1 work with Java 1.6 as well. Users are stronglyadvised to start moving to Java 1.8 (see HIVE-8607). 

•     Hadoop2.x (preferred), 1.x (not supported by Hive 2.0.0 onward).
Hive versions up to 0.13 also supported Hadoop 0.20.x, 0.23.x.

•     Hiveis commonly used in production Linux and Windows environment. Mac is a commonlyused development environment. The instructions in this document are applicableto Linux and Mac. Using it on Windows would require slightly differentsteps.  

HBase™  Requirements

This section lists required servicesand some required system configuration.

 

Table2. Java

HBase Version

JDK 6

JDK 7

JDK 8

1.2

Not Supported

yes

yes

1.1

Not Supported

yes

Running with JDK 8 will work but is not well tested.

1.0

Not Supported

yes

Running with JDK 8 will work but is not well tested.

0.98

yes

yes

Running with JDK 8 works but is not well tested. Building with JDK 8 would require removal of the deprecated remove() method of the PoolMap class and is under consideration. See HBASE-7608 for more information about JDK 8 support.

0.94

yes

yes

N/A

 

 

Hadoop  Requirements

Hadoop version support matrix

•     "S"= supported

•     "X"= not supported

•     "NT"= Not tested

 

 

HBase-0.94.x

HBase-0.98.x (Support for Hadoop 1.1+ is deprecated.)

HBase-1.0.x (Hadoop 1.x is NOT supported)

HBase-1.1.x

HBase-1.2.x

 

Hadoop-1.0.x

X

X

X

X

X

Hadoop-1.1.x

S

NT

X

X

X

Hadoop-0.23.x

S

X

X

X

X

Hadoop-2.0.x-alpha

NT

X

X

X

X

Hadoop-2.1.0-beta

NT

X

X

X

X

Hadoop-2.2.0

NT

S

NT

NT

X

Hadoop-2.3.x

NT

S

NT

NT

X

Hadoop-2.4.x

NT

S

S

S

S

Hadoop-2.5.x

NT

S

S

S

S

Hadoop-2.6.0

X

X

X

X

X

Hadoop-2.6.1+

NT

NT

NT

NT

S

Hadoop-2.7.0

X

X

X

X

X

Hadoop-2.7.1+

NT

NT

NT

NT

S

 

Pig™  Requirements

Unix and Windows users need thefollowing:

•     Hadoop 0.23.X, 1.X or 2.X- http://hadoop.apache.org/common/releases.html(You can run Pig with different versions of Hadoop by setting HADOOP_HOME topoint to the directory where you have installed Hadoop. If you do not setHADOOP_HOME, by default Pig will run with the embedded version, currentlyHadoop 1.0.4.)

•     Java 1.7- http://java.sun.com/javase/downloads/index.jsp(set JAVA_HOME to the root of your Java installation)

Optional

•     Python 2.7- https://www.python.org(when using Streaming Python UDFs)

•     Ant 1.8- http://ant.apache.org/(for builds)

Nutch 2.3.1  Requirements

The recommended Gora backends for thisNutch release are

•     ApacheAvro 1.7.6

•     ApacheHadoop 1.2.1 and 2.5.2

•     ApacheHBase 0.98.8-hadoop2 (although also tested with 1.X)

•     ApacheCassandra 2.0.2

•     ApacheSolr 4.10.3

•     MongoDB2.6.X

•     ApacheAccumlo 1.5.1

•     ApacheSpark 1.4.1

Nutch 2.3  Requirements

The supported Apache Gorav0.5 backends are;

;