Hadoop news

Hadoop


Top Stories

As the new year begins, global companies face the coming year's most prominent IT and business challenge: Big Data. The focus for IT will be to provide high performance analytics capabilities at the lowest cost, as business users need to tap into volumes of multi-structured data about their customers and markets to gain competitive advantage. RainStor, a provider of Big Data management software, has released five predictions focused on how enterprise Big Data will affect organizations in 2012. Based on client and partner experience, market research and conversations with industry experts, here are RainStor's five predictions for Big Data in 2012: Prediction #1: Big Data will Transition from Technology "Buzz" to a Real Business Challenge Affecting Many Large Global Enterprises Big Data is largely centered on leveraging the open source Apache Hadoop analytics platform... (more)

Big Moves in Big Data: EMC's Hadoop Strategy

To date, Big Storage has been locked out of Big Data. It’s been all about direct attached storage for several reasons. First, Advanced SQL players have typically optimized architectures from data structure (using columnar), unique compression algorithms, and liberal usage of caching to juice response over hundreds of terabytes. For the NoSQL side, it’s been about cheap, cheap, cheap along the Internet data center model: have lots of commodity stuff and scale it out. Hadoop was engineered exactly for such an architecture; rather than speed, it was optimized for sheer linear scale. Over the past year, most of the major platform players have planted their table stakes with Hadoop. Not surprisingly, IT household names are seeking to somehow tame Hadoop and make it safe for the enterprise. Up ' til now, anybody with armies of the best software engineers that Internet fir... (more)

Big Data Is All the Rage. Why?

On Monday, December 5, Bob Gourley went on the Enterprise CIO Forum to explain Big Data and why it matters. First, he defined Big Data simply as the data your organization cannot currently analyze. Though some technologists give more precise definitions, this sums up the challenge enterprises now face. If you can deal with all of your data now, you don’t have a Big Data problem, but as soon as you have more data than you can effectively manage to finding the answers you need fast enough to use them, you need a Big Data solution. Structured data and relational databases can also be Big Data but what we’re really talking about is the type and volume of information that exceeds traditional methods. New solutions include MapReduce, originally developed at Google to analyze and index the entire Internet, and Hadoop which grew to use those new methods. We see Big Data so... (more)

Cloud Computing and Big Data in 2013: What's Coming Next?

What changes in the cloud computing and big data landscape should we be expecting in 2013? In this article we offer a round-up of industry experts' opinions as they were asked by Cloud Expo / BigDataExpo Conference Chair Jeremy Geelan to preview the fast-approaching year ahead. 2013 Will Be The Year of Big Data  | The Internet of Things | Cloud To The Rescue (DR) | SSD John Engates | @jengates CTO of Rackspace Hosting Now its CTO, John joined Rackspace in August 2000, just a year after the company was founded, as VP of Operations, managing the datacenter operations and customer-service teams. Two years later, when Rackspace decided to add new services for larger enterprise customers, he created and helped develop the Intensive Hosting business unit. Most recently, he has played an active role in the evolution and evangelism of Rackspace's cloud computing strategy an... (more)

Employment Network Uses Big Data By @Dana_Gardner | @BigDataExpo #BigData

Employment Network Uses Big Data to Chart System Performance The next BriefingsDirect analytics innovation case study interview explores how Snagajob in Richmond, Virginia – one of the largest hourly employment networks for job seekers and employers – uses big data to finally understand their systems' performance in action. The result is vast improvement in how they provide rapid and richer services to their customers. Snagajob recently delivered 4 million new jobs applications in a single month through their systems. To learn how they're managing such impressive scale, BriefingsDirect sat down with Robert Fehrmann, Data Architect at Snagajob in Richmond, Virginia. The discussion is moderated by me, Dana Gardner, Principal Analyst at Interarbor Solutions. Here are some excerpts: Gardner: Tell us about your jobs matching organization. You’ve been doing this successfu... (more)

Oracle Database and Big Data: A Powerful Combination

Ever wondered how it is possible that Google searches through so much data with such speed and precision ? Part of the answer is MapReduce, Google technology for processing and generating large data sets. Apache Hadoop is open source software that can process petabytes of data in parallel on hundreds and thousands of commodity hardware nodes. It was inspired by Google MapReduce. Oracle corporation is acknowledging the power of Oracle/Hadoop combination by announcing Big Data Appliance - essentially Hadoop/Oracle database software/Oracle hardware bundle, to be available next year. Oracle database is an RDBMS which can be very slow when processing really big volumes of data. If tables become larger then couple of dozen GBs then you have to start using partitioning, index very carefully, get to know inner workings of query optimizer so that queries can be structured i... (more)

Accel Partners Launches $100m Big Data Fund to Spur Innovation in Data-Driven Technology

NEW YORK, Nov. 8, 2011 /PRNewswire/ -- Accel Partners, the global venture capital firm, has announced the launch of its Big Data Fund, a new $100m investment initiative created to identify innovative entrepreneurs seeking to build category-defining companies at every layer of the Big Data stack. Accel's Big Data Fund aims to fund transformative companies throughout the Big Data ecosystem, from next generation storage and data management platforms to a wide range of revolutionary software applications and services, including data analytics, vertical applications, mobile, and many more. Accel Partners will manage its Big Data Fund across global offices in U.S., London, China and India. "Big Data consistently drives innovation across our existing portfolio companies today and with the Big Data Fund we are empowering a new set of entrepreneurs to be able to catch t... (more)

Enabling Next Generation IO Architecture for Big Data at Cloud Expo NY

Cloud Expo New York $500 Savings here! More and more enterprises are deploying cloud computing and IT departments are requiring new capabilities with cloud storage infrastructure. Hadoop and Map Reduce have been the popular applications for handling distributed unstructured data. In his session at the 10th International Cloud Expo, Ilker Cebeli, Director of Cloud Computing at QLogic, will explain how Big Data is already impacting storage and networking architecture in the data centers and how to build optimized storage and networking infrastructure for the future. He will also explain how to leverage cloud storage while maintaining iSCSI and Fibre Channel local storage infrastructure. Explore Cloud Expo Sponsorship & Exhibit Opportunities ! Speaker Bio: Ilker Cebeli is Director of Cloud Computing at QLogic. He is responsible for leading and directing the cloud stra... (more)

Big Data Analytics: Datameer

Datameer provides a big data solution that focuses on perhaps the most important niche in this growing domain, the end-user. Here is an introduction from their website: Datameer leverages the scalability, flexibility and cost-effectiveness of Apache Hadoop to deliver an end-user focused analytics platform for big data. Datameer overcomes Hadoop’s complexity and lack of tools by providing analysts and technical users with pre-built functionality across data integration, analytics and data visualization of any volume and types of data. I’ve met with the CEO (Stefan Groschupf) and other Datameer executives. I’ve also interacted with them in events like our Government Big Data Forum. Through these events plus demonstrations by some of their greatest engineers has led me to a few conclusions about Datameer. In general, I believe enterprise technologists should take not... (more)

The Big Data Revolution

For many years, companies collected data from various sources that often found its way into relational databases like Oracle and MySQL. However, the rise of the Internet, Web 2.0, and recently social media began an enormous increase in the amount of data created as well as in the type of data. No longer was data relegated to types that easily fit into standard data fields. Instead, it now came in the form of photos, geographic information, chats, Twitter feeds, and emails. The age of Big Data is upon us. Big Data Beginnings A study by IDC titled "The Digital Universe Decade" projects a 45-fold increase in annual data by 2020. In 2010, the amount of digital information was 1.2 zettabytes (1 zettabyte equals 1 trillion gigabytes). To put that in perspective, the equivalent of 1.2 zettabytes is a full-length episode of "24" running continuously for 125 million years, ac... (more)

Examining the True Cost of Big Data

The good news about the Big Data market is that we generally all agree on the definition of Big Data, which has come to be known as data that has volume, velocity and variety where businesses need to collect, store, manage and analyze in order to derive business value or otherwise known as the "4 V's." However, the problem with such a broad definition is that it can mean different things to different people once you start to put some real values next to those V's. Let's be honest, Volume can be a different thing to different organizations. To some it is anything above 10 terabytes of managed data in their BI environment and to others it is petabyte scale and nothing less. Likewise velocity can be multi-billions of daily records coming into the enterprise from various external and internal networks. When it really comes down to it, each business situation will be qu... (more)