Hadoop news


Top Stories

When we last left Oracle’s big data plans, there was definitely a missing piece. Oracle’s Big Data Appliance as initially disclosed at last fall’s OpenWorld was a vague plan that appeared to be positioned primarily as an appliance that would accompany and feed data to Exadata. Oracle did specify some utilities, such as an enterprise version of the open source R statistical processing program that was designed for multithreaded execution, plus a distribution of a NoSQL database based on Oracle’s BerkeleyDB as an alternative to Apache Hive. But the emphasis appeared to be extraction and transformation of data for Exadata via Oracle’s own utilities that were optimized for its platform. With Oracle’s announcement of general availability of the big data appliance, it is filling in the blanks. As such, Oracle’s plan for Hadoop was competition, not for Cloudera (or Horto... (more)

Cloud Computing and Big Data in 2013: What's Coming Next?

What changes in the cloud computing and big data landscape should we be expecting in 2013? In this article we offer a round-up of industry experts' opinions as they were asked by Cloud Expo / BigDataExpo Conference Chair Jeremy Geelan to preview the fast-approaching year ahead. 2013 Will Be The Year of Big Data  | The Internet of Things | Cloud To The Rescue (DR) | SSD John Engates | @jengates CTO of Rackspace Hosting Now its CTO, John joined Rackspace in August 2000, just a year after the company was founded, as VP of Operations, managing the datacenter operations and customer-service teams. Two years later, when Rackspace decided to add new services for larger enterprise customers, he created and helped develop the Intensive Hosting business unit. Most recently, he has played an active role in the evolution and evangelism of Rackspace's cloud computing strategy an... (more)

Big Data Contributes to Public Safety: Hadoop for Law Enforcement

CTOlabs.com, a subsidiary of the technology research, consulting and services firm Crucial Point LLC and a peer site of CTOvision.com, has just published a white paper providing context and use cases on Hadoop For Law Enforcement, an important mission-focused domain ripe for the application of more Big Data solutions. From the report: Big Data, the data too large and complex for your current information infrastructure to store and analyze, has changed every sector in government and industry. Today’s sensors and devices produce an overwhelming amount of information that is often unstructured, and solutions developed to handle Big Data now allowing us to track more information and run more complex analytics to gain a level of insight once thought impossible. The dominant Big Data solution is the Apache Hadoop ecosystem which provides an open source platform for reli... (more)

Big Data Conundrum: Show Me the Money!

Inventory levels. Sales results. Negative comments on Facebook. Positive comments on Twitter. Shopping on Amazon. Listening to Pandora. Online search habits. No matter what you call it or what the information describes, it’s all data being collected about you. Thanks to new technologies like Hadoop, once-unquantifiable data (like Facebook conversations and Tweets) can now be quantified. Now, because nearly everything is measurable, everything is measured. The result: companies are spending big dollars to collect, store and measure astronomical amounts of data. Show me the data! There’s a name for this movement: Big Data. Not only is it a name, it has been the “it, it” of 2012, possibly trumping “the cloud.” IDC defines Big Data as projects collecting 100 terabytes of data (hence the name), comprising two or more data formats. Earlier this year, the research firm ... (more)

In 2014 Big Data Investments Will Account for Nearly $30 Billion - Eventually Accounting for $76 Billion by 2020 End

DALLAS, Aug. 21, 2014 /PRNewswire-iReach/ -- Amid the proliferation of real time data from sources such as mobile devices, web, social media, sensors, log files and transactional applications, Big Data has found a host of vertical market applications, ranging from fraud detection to R&D. Photo - http://photos.prnewswire.com/prnh/20140821/138541 "Big Data Market: 2014 – 2020 – Opportunities, Challenges, Strategies, Industry Verticals & Forecasts" Key Findings: In 2014 Big Data vendors will pocket nearly $30 Billion from hardware, software and professional services revenues Big Data investments are further expected to grow at a CAGR of nearly 17% over the next 6 years, eventually accounting for $76 Billion by the end of 2020 The market is ripe for acquisitions of pure-play Big Data startups, as competition heats up between IT incumbents Nearly every large scale IT ven... (more)

Oracle Database and Big Data: A Powerful Combination

Ever wondered how it is possible that Google searches through so much data with such speed and precision ? Part of the answer is MapReduce, Google technology for processing and generating large data sets. Apache Hadoop is open source software that can process petabytes of data in parallel on hundreds and thousands of commodity hardware nodes. It was inspired by Google MapReduce. Oracle corporation is acknowledging the power of Oracle/Hadoop combination by announcing Big Data Appliance - essentially Hadoop/Oracle database software/Oracle hardware bundle, to be available next year. Oracle database is an RDBMS which can be very slow when processing really big volumes of data. If tables become larger then couple of dozen GBs then you have to start using partitioning, index very carefully, get to know inner workings of query optimizer so that queries can be structured i... (more)

How Enterprise Big Data Will Affect Organizations in 2012

As the new year begins, global companies face the coming year's most prominent IT and business challenge: Big Data. The focus for IT will be to provide high performance analytics capabilities at the lowest cost, as business users need to tap into volumes of multi-structured data about their customers and markets to gain competitive advantage. RainStor, a provider of Big Data management software, has released five predictions focused on how enterprise Big Data will affect organizations in 2012. Based on client and partner experience, market research and conversations with industry experts, here are RainStor's five predictions for Big Data in 2012: Prediction #1: Big Data will Transition from Technology "Buzz" to a Real Business Challenge Affecting Many Large Global Enterprises Big Data is largely centered on leveraging the open source Apache Hadoop analytics platform... (more)

Cousins of Cobol in Big Data Analytics

In this  article  I would  like to look at a few tools which are overlooked when it comes to Big Data analytics. Organizations that  have  already  heavy investment  on Mainframe  and  would like to continue  with the utilization of Mainframe can consider these  tools for further  expanding their Big Data Analytics reach. DFSORT-  Sorting & Merging Large Data Sets : Much before RDBMS have taken their place, Cobol programs have 2 major file manipulation operations namely: SORT operation accepts un-sequenced input and produces output in specified sequence The Merge operation compares records from two or more files and combines them in order DFSORT adds the ability to do faster and easier sorting, merging, copying, reporting and analysis of your business information, as well as versatile data handling at the record, fixed position/length or variable position/length fi... (more)

The Big Data Revolution

For many years, companies collected data from various sources that often found its way into relational databases like Oracle and MySQL. However, the rise of the Internet, Web 2.0, and recently social media began an enormous increase in the amount of data created as well as in the type of data. No longer was data relegated to types that easily fit into standard data fields. Instead, it now came in the form of photos, geographic information, chats, Twitter feeds, and emails. The age of Big Data is upon us. Big Data Beginnings A study by IDC titled "The Digital Universe Decade" projects a 45-fold increase in annual data by 2020. In 2010, the amount of digital information was 1.2 zettabytes (1 zettabyte equals 1 trillion gigabytes). To put that in perspective, the equivalent of 1.2 zettabytes is a full-length episode of "24" running continuously for 125 million years, ac... (more)

Little Data, Big Data and Very Big Data (VBD) or Big BS?

This is an industry trends and perspective piece about big data and little data, industry adoption and customer deployment. If you are in any way associated with information technology (IT), business, scientific, media and entertainment computing or related areas, you may have heard big data mentioned. Big data has been a popular buzzword bingo topic and term for a couple of years now. Big data is being used to describe new and emerging along with existing types of applications and information processing tools and techniques. I routinely hear from different people or groups trying to define what is or is not big data and all too often those are based on a particular product, technology, service or application focus. Thus it should be no surprise that those trying to police what is or is not big data will often do so based on what their interest, sphere of influence,... (more)

Big Data in Financial Analytics

Big Data & Text Analytics: As  the analysis  of  large amounts  of unstructured  data is gaining a major space in enterprise  computing,  we  are seeing the emergence of more use cases in this regard.  While  the  term   "Big"  in Big Data   makes it more synonymous  with  Massively Parallel Processing frameworks like Hadoop,  however  the  underlying the success of  Big Data  relies  on effective usage of  content analytics  of the underlying  unstructured data.  I have high lighted  this thought process in my earlier  article, Big Data Analytics Thinking Outside Of Hadoop. Unstructured Content Analytics is  defined  as the   process of  gaining  new insights  from  the  unstructured data, by  employing   text mining, image recognition, voice recognition and other related analytical techniques. Big Data Journal was launched on SYS-CON.com in 2012 The below  mater... (more)