Hadoop news


Top Stories

Big data capabilities and advanced business analytics have now become essential to nearly any business development activity. The benefits that enterprises can get if they can get their hands around big data analytics and apply it to business challenges are quickly being documented -- and they come as big new profits and major market advantages. Industries around the world are rapidly seeking transformational projects using big data to gain competitive advantage. As part of the next edition of the HP Big Data Podcast Series, BriefingsDirect sat down with two HP executives to learn how these advanced analytics seekers can best accomplish their goals. The insights gleaned include how companies worldwide are best capturing myriad knowledge, gaining ever deeper analysis, and rapidly and securely making those insights available to more people on their own terms. So join ... (more)

Cloud Computing and Big Data in 2013: What's Coming Next?

What changes in the cloud computing and big data landscape should we be expecting in 2013? In this article we offer a round-up of industry experts' opinions as they were asked by Cloud Expo / BigDataExpo Conference Chair Jeremy Geelan to preview the fast-approaching year ahead. 2013 Will Be The Year of Big Data  | The Internet of Things | Cloud To The Rescue (DR) | SSD John Engates | @jengates CTO of Rackspace Hosting Now its CTO, John joined Rackspace in August 2000, just a year after the company was founded, as VP of Operations, managing the datacenter operations and customer-service teams. Two years later, when Rackspace decided to add new services for larger enterprise customers, he created and helped develop the Intensive Hosting business unit. Most recently, he has played an active role in the evolution and evangelism of Rackspace's cloud computing strategy an... (more)

2015 Big Data Startups By @JimKaskade | @BigDataExpo #BigData

The following are a few  Big Data companies that I was keeping an eye on starting back in January of 2015  (yeah, this is NOT a statistically significant sample)....some for "old times sake" because I knew the founding team and was curious how things were progressing, some because I trust their venture capital partners, and some because they inspire me with an application of the technology that is meaningful (I'm tired of hearing about yet another "Big Data Platform"). Adatao Founder(s): Christopher Nguyen, Mike Bui, Tony Do Headquarters: Sunnyvale, CA Website: www.adatao.com Type of business: Data Intelligence for All Adatao is Big Data 2.0, enabling the convergence of business intelligence, data science and machine learning directly on top of big data. Adatao is leading the Big Data 2.0 charge by making it easy for business users, data scientists and engineers to co... (more)

In 2014 Big Data Investments Will Account for Nearly $30 Billion - Eventually Accounting for $76 Billion by 2020 End

DALLAS, Aug. 21, 2014 /PRNewswire-iReach/ -- Amid the proliferation of real time data from sources such as mobile devices, web, social media, sensors, log files and transactional applications, Big Data has found a host of vertical market applications, ranging from fraud detection to R&D. Photo - http://photos.prnewswire.com/prnh/20140821/138541 "Big Data Market: 2014 – 2020 – Opportunities, Challenges, Strategies, Industry Verticals & Forecasts" Key Findings: In 2014 Big Data vendors will pocket nearly $30 Billion from hardware, software and professional services revenues Big Data investments are further expected to grow at a CAGR of nearly 17% over the next 6 years, eventually accounting for $76 Billion by the end of 2020 The market is ripe for acquisitions of pure-play Big Data startups, as competition heats up between IT incumbents Nearly every large scale IT ven... (more)

Oracle Fills Another Gap in Its Big Data Offering

When we last left Oracle’s big data plans, there was definitely a missing piece. Oracle’s Big Data Appliance as initially disclosed at last fall’s OpenWorld was a vague plan that appeared to be positioned primarily as an appliance that would accompany and feed data to Exadata. Oracle did specify some utilities, such as an enterprise version of the open source R statistical processing program that was designed for multithreaded execution, plus a distribution of a NoSQL database based on Oracle’s BerkeleyDB as an alternative to Apache Hive. But the emphasis appeared to be extraction and transformation of data for Exadata via Oracle’s own utilities that were optimized for its platform. With Oracle’s announcement of general availability of the big data appliance, it is filling in the blanks. As such, Oracle’s plan for Hadoop was competition, not for Cloudera (or Horto... (more)

Big Data Conundrum: Show Me the Money!

Inventory levels. Sales results. Negative comments on Facebook. Positive comments on Twitter. Shopping on Amazon. Listening to Pandora. Online search habits. No matter what you call it or what the information describes, it’s all data being collected about you. Thanks to new technologies like Hadoop, once-unquantifiable data (like Facebook conversations and Tweets) can now be quantified. Now, because nearly everything is measurable, everything is measured. The result: companies are spending big dollars to collect, store and measure astronomical amounts of data. Show me the data! There’s a name for this movement: Big Data. Not only is it a name, it has been the “it, it” of 2012, possibly trumping “the cloud.” IDC defines Big Data as projects collecting 100 terabytes of data (hence the name), comprising two or more data formats. Earlier this year, the research firm ... (more)

Big Data – Big Help or Big Risk?

By Andy Thurai (Twitter: @AndyThurai) (Original version of this blog appeared on ProgrammableWeb) As promised in my last blog “Big Data, API, and IoT …..Newer technologies protected by older security” here is a deep dive on Big Data security and how to effortlessly secure Big Data effectively. Like many other open source models, Hadoop has followed a path that hasn’t focused much on security.  In order to effectively use Big Data, it needs to be secured properly. However if you try to force fit into an older security model, you might end up compromising more than you think. But if you make it highly secure, it might interfere with performance. In order to effectively secure Big Data, you must mitigate the following security risks that aren’t addressed by prior security models. Issue #1:  Are the keys to the kingdom with you? In a hosted environment, the provider holds ... (more)

Big Data Predictions for 2016 By @JnanDash | @BigDataExpo #BigData

As every year begins, several experts and analyst firms like to make predictions. Let us try to make some observations in an area much talked about lately – Big Data. So here goes: The Big Data quandary will continue as companies try to understand its value to business. Just dumping all kinds of data into a data lake (read Hadoop) is not going to solve anything. There has to be business value on what insights are needed. Therefore much like the Data Warehousing era brought additional tools in the ETL space, there is need for data curation and transformation for practical use besides the analytics piece. Demand for BI and Analytics will reach new heights. The next-generation BI and analytics platform should help business tap into the power of their data, whether in the cloud or on-premises. This ‘Networked BI’ capability creates an interwoven data fabric that delivers ... (more)

Oracle Database and Big Data: A Powerful Combination

Ever wondered how it is possible that Google searches through so much data with such speed and precision ? Part of the answer is MapReduce, Google technology for processing and generating large data sets. Apache Hadoop is open source software that can process petabytes of data in parallel on hundreds and thousands of commodity hardware nodes. It was inspired by Google MapReduce. Oracle corporation is acknowledging the power of Oracle/Hadoop combination by announcing Big Data Appliance - essentially Hadoop/Oracle database software/Oracle hardware bundle, to be available next year. Oracle database is an RDBMS which can be very slow when processing really big volumes of data. If tables become larger then couple of dozen GBs then you have to start using partitioning, index very carefully, get to know inner workings of query optimizer so that queries can be structured i... (more)

Big Data Is All the Rage. Why?

On Monday, December 5, Bob Gourley went on the Enterprise CIO Forum to explain Big Data and why it matters. First, he defined Big Data simply as the data your organization cannot currently analyze. Though some technologists give more precise definitions, this sums up the challenge enterprises now face. If you can deal with all of your data now, you don’t have a Big Data problem, but as soon as you have more data than you can effectively manage to finding the answers you need fast enough to use them, you need a Big Data solution. Structured data and relational databases can also be Big Data but what we’re really talking about is the type and volume of information that exceeds traditional methods. New solutions include MapReduce, originally developed at Google to analyze and index the entire Internet, and Hadoop which grew to use those new methods. We see Big Data so... (more)

Big Moves in Big Data: EMC's Hadoop Strategy

To date, Big Storage has been locked out of Big Data. It’s been all about direct attached storage for several reasons. First, Advanced SQL players have typically optimized architectures from data structure (using columnar), unique compression algorithms, and liberal usage of caching to juice response over hundreds of terabytes. For the NoSQL side, it’s been about cheap, cheap, cheap along the Internet data center model: have lots of commodity stuff and scale it out. Hadoop was engineered exactly for such an architecture; rather than speed, it was optimized for sheer linear scale. Over the past year, most of the major platform players have planted their table stakes with Hadoop. Not surprisingly, IT household names are seeking to somehow tame Hadoop and make it safe for the enterprise. Up ' til now, anybody with armies of the best software engineers that Internet fir... (more)