Hadoop news


Top Stories

Addressing the Ethical Issues of Big Data By Rick Delgado Big Data is a growing trend, making it possible for businesses of every kind to gather, store, and use the unlimited personal and private data found on the internet. This massive collection allows them to predict trends, determine consumer tastes, and hone marketing plans for the greatest success. However, the concept of collecting vast amounts of information on people sounds dangerous all on its own; what keeps businesses from stepping over the line? What ethics code ensures consumer privacy along with corporate profit? Here are three of the main concerns about Big Data’s ethics, and ways to combat the issues. 1. Working in Secret "Big brother is watching,” in the modern age is quickly becoming “Big Data is watching.” Many people fear that the invention of Big Data and the advantages it offers have encouraged... (more)

Big Moves in Big Data: EMC's Hadoop Strategy

To date, Big Storage has been locked out of Big Data. It’s been all about direct attached storage for several reasons. First, Advanced SQL players have typically optimized architectures from data structure (using columnar), unique compression algorithms, and liberal usage of caching to juice response over hundreds of terabytes. For the NoSQL side, it’s been about cheap, cheap, cheap along the Internet data center model: have lots of commodity stuff and scale it out. Hadoop was engineered exactly for such an architecture; rather than speed, it was optimized for sheer linear scale. Over the past year, most of the major platform players have planted their table stakes with Hadoop. Not surprisingly, IT household names are seeking to somehow tame Hadoop and make it safe for the enterprise. Up ' til now, anybody with armies of the best software engineers that Internet fir... (more)

Big Data Analytics: Thinking Outside of Hadoop

Big Data Predictions In the recent release of '2012 Hype Cycle Of Emerging Technologies,' research analyst Gartner evaluated several technologies to come up with a list of technologies that will dominate the future . "Big Data" related technologies form a significant portion of the list, in particular the following technologies revolve around the concept and usage of Big Data. Social Analytics: This analytics allow marketers to identify sentiment and identify trends in order to accommodate the customer better. Activity Streams: Activity Streams are the future of enterprise collaboration, uniting people, data, and applications in real-time in a central, accessible, virtual interface. Think of a company social network where every employee, system, and business process exchanged up-to-the-minute information about their activities and outcomes Natural Language Question A... (more)

Big Data and Visualization Give Enterprises New Views By @Dana_Gardner | @BigDataExpo #BigData

Big Data and Visualization Give Enterprises New Views into Their Business The next BriefingsDirect big data innovation discussion highlights how Tableau Software and big data analytics platforms come together to provide visualization benefits for those seeking more than just crunched numbers. They're looking for ways to improve their businesses effectively and productively, and to share the analysis quickly and broadly. To learn more, BriefingsDirect sat down with Paul Lilford, Global Director of Technology Partners for Tableau Software, based in Seattle, and Steve Murfitt, Director of Technical Alliances at HP Vertica. The discussion is moderated by me, Dana Gardner, Principal Analyst at Interarbor Solutions. Here are some excerpts: Gardner: Why is the tag-team between Tableau and big data so popular. Every time I speak with some one using Vertica, they inevitably me... (more)

Cousins of Cobol in Big Data Analytics

In this  article  I would  like to look at a few tools which are overlooked when it comes to Big Data analytics. Organizations that  have  already  heavy investment  on Mainframe  and  would like to continue  with the utilization of Mainframe can consider these  tools for further  expanding their Big Data Analytics reach. DFSORT-  Sorting & Merging Large Data Sets : Much before RDBMS have taken their place, Cobol programs have 2 major file manipulation operations namely: SORT operation accepts un-sequenced input and produces output in specified sequence The Merge operation compares records from two or more files and combines them in order DFSORT adds the ability to do faster and easier sorting, merging, copying, reporting and analysis of your business information, as well as versatile data handling at the record, fixed position/length or variable position/length fi... (more)

Examining the True Cost of Big Data

The good news about the Big Data market is that we generally all agree on the definition of Big Data, which has come to be known as data that has volume, velocity and variety where businesses need to collect, store, manage and analyze in order to derive business value or otherwise known as the "4 V's." However, the problem with such a broad definition is that it can mean different things to different people once you start to put some real values next to those V's. Let's be honest, Volume can be a different thing to different organizations. To some it is anything above 10 terabytes of managed data in their BI environment and to others it is petabyte scale and nothing less. Likewise velocity can be multi-billions of daily records coming into the enterprise from various external and internal networks. When it really comes down to it, each business situation will be qu... (more)

Big Data Conundrum: Show Me the Money!

Inventory levels. Sales results. Negative comments on Facebook. Positive comments on Twitter. Shopping on Amazon. Listening to Pandora. Online search habits. No matter what you call it or what the information describes, it’s all data being collected about you. Thanks to new technologies like Hadoop, once-unquantifiable data (like Facebook conversations and Tweets) can now be quantified. Now, because nearly everything is measurable, everything is measured. The result: companies are spending big dollars to collect, store and measure astronomical amounts of data. Show me the data! There’s a name for this movement: Big Data. Not only is it a name, it has been the “it, it” of 2012, possibly trumping “the cloud.” IDC defines Big Data as projects collecting 100 terabytes of data (hence the name), comprising two or more data formats. Earlier this year, the research firm ... (more)

Big Data Predictions for 2013

My prediction for 2013 is that competitive advantage will translate into enterprises using sophisticated Big Data analytics to create a new breed of applications - Intelligent Applications. "It's more than just insights from MapReduce", a CIO from a fortune 100 told me, "It's about using data to make our customer touch points more engaging, more interactive, more intelligent." So when you hear about "Big Data solutions," you need to translate that into a new category of "Intelligent Applications." At the end of the day, it's not about people pouring through petabytes of data. It's actually about how one turns the data into revenue (or profits). This means that you MUST: Start with the business problem first (preferably one with revenue upside versus cost savings) Determine which data elements you can leverage AFTER #1 Define an analytical three-tier architecture (a... (more)

Infochimps Shares the Secrets of Big Data Success

AUSTIN, Texas, May 22, 2013 /PRNewswire/ -- Big Data cloud service provider Infochimps (Infochimps.com) announced today the release of "How to Do a Big Data Project," a step by step guide for large enterprises implementing Big Data projects. Recent research by Infochimps has shown that nearly half of all Big Data projects fail. With this new guide, now available for download, the company hopes to improve this success rate. Infochimps is offering insight into some of its own best practices for deriving business value from Big Data with an open-standards architecture and strategy, which it has honed working with some of the world's top companies. Big Data: Boon and Bane of Enterprise IT As Big Data sweeps the business world, there's broad consensus on its value, but no standard approach for following a project through from inception to completion. Research has shown th... (more)

Big Data Analysis Boosts User Experience for Spirent By @Dana_Gardner | @BigDataExpo #BigData

Transcript of a discussion on the use of big data to provide improved user experiences for telecommunications operators' customers. Dana Gardner: Hello, and welcome to the next edition of the HP Discover Podcast Series. I'm Dana Gardner, Principal Analyst at Interarbor Solutions, your host and moderator for this ongoing discussion on IT innovation and how it’s making an impact on people’s lives. Gardner Our next big-data case study discussion explores the ways that Spirent Communications advances the use of big data to provide improved user experiences for telecommunications operators. We'll learn how advanced analytics that draws on multiple data sources provide Spirent’s telco customers’ rapid insights into their networks and operations.  That insight, combined with analysis of user actions and behaviors, provides a "total picture" approach to telco services and ... (more)

INOVVO Uses Big Data for Customer Satisfaction By @Dana_Gardner | @BigDataExpo #BigData

The next BriefingsDirect big-data case study discussion examines how INOVVO delivers impactful network analytical services for mobile operators to help them engender improved end-user loyalty. We'll see how advanced analytics, drawing on multiple data sources, enables INOVVO’s mobile carrier customers to provide mobile users with faster, more reliable, and relevant services. To learn more about how INOVVO uses big data to make major impacts on mobile services, please join me in welcoming Joseph Khalil, President and CEO of INOVVO in Reston, Virginia. The discussion is moderated by me, Dana Gardner, Principal Analyst at Interarbor Solutions. Here are some excerpts: Gardner: User experience and quality of service are so essential nowadays. What has been the challenge for you to gain an integrated and comprehensive view of subscribers and networks that they're on in or... (more)