Hadoop权威指南（影印版） 下载 pdf 百度网盘 epub 免费 2025 电子书 mobi 在线

揭示了Apache

Hadoop如何为你释放数据的力量。这本内容全面的书籍展示了如何使用Hadoop架构搭建和维护可靠、可伸缩的分布式系统。Hadoop架构是MapReduce算法的一种开源应用，是Google开创其帝国的重要基石。程序员可从中探索如何分析海量数据集，管理员可以了解如何建立与运行Hadoop集群。

《Hadoop权威指南(影印版第2版修订版)》涵盖了Hadoop近的更新，包括诸如Hive、Sqoop和Avro之类的新特性。它也提供了案例学习来展示Hadoop如何解决特殊问题。期待尽情享受你的数据？这就是你要的书。本身由Tom

White著。

Foreword

Preface

1. Meet Hadoop

Data!

Data Storage and Analysis

Comparison with Other Systems

RDBMS

Grid Computing

Volunteer Computing

A Brief History of Hadoop

Apache Hadoop and the Hadoop Ecosystem

2. MapReduce

A Weather Dataset

Data Format

Analyzing the Data with Unix Tools

Analyzing the Data with Hadoop

Map and Reduce

Java MapReduce

Scaling Out

Data Flow

Combiner Functions

Running a Distributed MapReduce Job

Hadoop Streaming

Ruby

Python

Hadoop Pipes

Compiling and Running

3. The Hadoop Distributed Filesystem

The Design of HDFS

HDFS Concepts

Blocks

Namenodes and Datanodes

The Command-Line Interface

Basic Filesystem Operations

Hadoop Filesystems

Interfaces

The Java Interface

Reading Data from a Hadoop URL

Reading Data Using the FileSystem API

Writing Data

Directories

Querying the Filesystem

Deleting Data

Data Flow.

Anatomy of a File Read

Anatomy of a File Write

Coherency Model

Parallel Copying with distcp

Keeping an HDFS Cluster Balanced

Hadoop Archives

Using Hadoop Archives

Limitations

4. Hadoop I/0

Data Integrity

Data Integrity in HDFS

LocalFileSystem

ChecksumFileSystem

Compression

Codecs

Compression and Input Splits

Using Compression in MapReduce

Serialization

The Writable Interface

Writable Classes

Implementing a Custom Writable

Serialization Frameworks

Avro

File-Based Data Structures

SequenceFile

……

     Tom White从2007年起就是Apache

Hadoop的理事。他是Apache软件基金会的成员和Cloudera的工程师。Tom为oreilly．com，java．net~llBM的developerWorks撰文，并为业内会议演讲。

暂无出版社相关信息，正在全力查找中！

暂无相关书籍摘录，正在全力查找中！

在线阅读地址：Hadoop权威指南（影印版）在线阅读

在线听书地址：Hadoop权威指南（影印版）在线收听

在线购买地址：Hadoop权威指南（影印版）在线购买

* The architecture of HDFS is described in “The Hadoop Distributed File System” by Konstantin Shvachko,

Hairong Kuang, Sanjay Radia, and Robert Chansler (Proceedings of MSST2010, May 2010, http://

storageconference.org/2010/Papers/MSST/Shvachko.pdf).

† “Scaling Hadoop to 4000 nodes at Yahoo!,” http://developer.yahoo.net/blogs/hadoop/2008/09/scaling_hadoop

_to_4000_nodes_a.html.

在许多情况下，可以视Mapreduce为关系型数据库管理系统的补充。MapReduce比较适合以批处理的方式处理需要分析整个数据集的问题，尤其是即席分析。RDBMS适用于点查询和更新，数据集被索引后，数据库系统能够提供低延迟的数据检索和快速的少量数据更新。MapReduce适合数据一次写入、多次读取的应用，而关系型数据库更适合持续更新数据集.

MapReduce is a programming model for data processing. MapReduce works by breaking the processing into two phases: the map phase and the reduce phase. Each phase has key-value pairs as input and output, the types of which may be chosen by the programmer. The programmer also specifies two functions: the map function and the reduce function.

Hadoop divides the input to a MapReduce job into fixed-size pieces called input splits, or just splits. Hadoop creates one map task for each split, which runs the userdefined map function for each record in the split. Hadoop does its best to run the map task on a node where the input data resides in HDFS. This is called the data locality optimization.

When there are multiple reducers, the map tasks partition their output, each creating one partition for each reduce task. There can be many keys (and their associated values) in each partition, but the records for every key are all in a single partition. The partitioning can be controlled by a user-defined partitioning function, but normally the default partitioner—which buckets keys using a hash function—works very well.

HDFS is a filesystem designed for storing very large files with streaming data access patterns(write-once, read-many-times pattern), running on clusters of commodity hardware.

HDFS blocks(>64M) are large compared to disk blocks, and the reason is to minimize the cost of seeks. Map tasks in MapReduce normally operate on one block at a time, so if you have too few tasks (fewer than nodes in the cluster), your jobs will run slower than they could otherwise.

An HDFS cluster has two types of node operating in a master-worker pattern: a namenode (the master) and a number of datanodes (workers). The namenode manages the filesystem namespace. It maintains the filesystem tree and the metadata for all the files and directories in the tree. Datanodes are the workhorses of the filesystem. They store a...

One important aspect of this design is that the client contacts datanodes directly to retrieve data and is guided by the namenode to the best datanode for each block. This design allows HDFS to scale to a large number of concurrent clients, since the data traffic is spread across all the datanodes in the cluster.

Hadoop takes a simple approach in which the network is represented as a tree and the distance between two nodes is the sum of their distances to their closest common ancestor.

书籍介绍

《Hadoop权威指南（影印版）（第2版修订版）》揭示了ApacheHadoop如何为你释放数据的力量。这本内容全面的书籍展示了如何使用Hadoop架构搭建和维护可靠、可伸缩的分布式系统。Hadoop架构是MapReduce算法的一种开源应用，是Google开创其帝国的重要基石。程序员可从中探索如何分析海量数据集，管理员可以了解如何建立与运行Had00p集群。本修订版涵盖了Hadoop最近的更新，包括诸如Hive、Sqoop和Avr0之类的新特性。它也提供了案例学习来展示Hadoop如何解决特殊问题。期待尽情享受你的数据？这就是你要的书。