Writing software that can simultaneously make use of multiple processors can be hard. Yet the advent of multicore processors--four cores per chip is now common--means that more and more software needs to do just that.
With processor performance increases now increasingly coming through the ability to handle more execution threads, rather than handling individual ones faster, multithreaded programming, in one form or another, is pretty much the only path to writing faster software, going forward.
Pervasive DataRush architecture.
(Credit: Pervasive Software)Researchers and developers are tackling this issue from a lot of different angles, including new languages and a greater focus on multithreaded programming in computer science curricula. However, perhaps the most promising general direction is toward what you might call multicore virtualization--the abstraction of parallel complexities by carefully crafted algorithms and run times that handle most of the heavy lifting. (MapReduce and gaming engines are examples of the sort of thing I'm talking about.)
The latest announcement in this vein comes from Pervasive Software, whose DataRush product is now generally available. The company describes DataRush thusly:
At the heart of Pervasive DataRush is a powerful, massively parallel data-processing engine that enables fast, efficient, deep analysis and searching of large data stores. The platform integrates breakthrough technology to resolve well-known parallel-programming challenges associated with writing software for multicore processors: built-in features automatically handle issues such as locking, threading, and deadlock.
From a technical perspective, DataRush is a library and processing engine written in Java--which makes it portable to different operating systems because it runs in a Java Virtual Machine, as opposed to directly on the operating system. It's not specific to any one application, and it has operators and application programming interfaces (APIs) that can be exploited for a variety of parallel applications.
That said, Pervasive has focused and optimized DataRush around analytic tasks that typically require lots of parallel processing. A typical application is the sort of near-real-time data crunching that credit card companies do to detect and counter fraud.
From a business perspective, Pervasive's strategy with DataRush is to move up the software stack to higher-level solutions. However, these will still be in the form of enabling applications to handle certain types of tasks rather than actually getting into the application space themselves.
Pervasive Software is a well-established software company that has been listed on Nasdaq since 1997 and counts thousands of users as its customers. Along with a set of data integration products, it sells a database that is popular with many "low IT" organizations (PDF). Now it's adding DataRush to tackle a problem that didn't even exist for the mainstream users that Pervasive serviced 10 years ago.
I spent the past couple of days attending Technology Review's EmTech08 conference at MIT. Lots of interesting speakers and ideas, some in areas of tech that I follow day-to-day (such as cloud computing) and others that I follow more in the vein of an interested observer (alternative fuels, open voting systems). In many respects, it's a refreshing change of pace from the events I commonly attend that tend to be more focused on today's immediate IT concerns.
EmTech08 gave me lots to mull--and I'll roll that mulling into more in-depth pieces down the road. For today, though, I'm just going to expand a bit on a few statements and thoughts I ran across in the course of the two days that particularly caught my attention.
The state of the market for tools in parallel computing is abysmal. (Marc Snir, University of Illinois)
There seems to be a default assumption around the IT industry today that, as processors evolve to more cores and more heterogeneous processing (and as computing architectures get more distributed), the software will evolve apace. Changes will be required, of course, but nothing to really worry about. I'm not so sure. Even if one discounts the most dramatic doomsayers, lots of researchers and IT executives see serious gaps in both tools and training to deal with highly threaded processing. Consider that several of the panelists in the Parallel Programming session spoke warmly of the potential for functional languages. Yet it's very early days for the likes of Haskell and Fortress and, in general, language development and adoption is a very long process.
People choose killer apps. (Craig Mundie, Microsoft)
Craig Mundie spoke at length about many of the characteristics that he saw such an application as having. He used terms like context- and location-aware, immersive, and personal. The general theme was "client + cloud," the idea being that this level of realism and immersion requires huge processing power and function at the client even if the data and orchestration takes place in the network someplace. He didn't really specify a specific next-generation application though. I'm not sure killer "app" is really the right term though. The original PC had a true killer app--the spreadsheet. But subsequent generations have really had interaction models--first the GUI and then the browser. The next generation will probably be something similar, exposing an even more varied set of applications but in richer ways. (CNET News' Dan Farber has an in-depth post on Mundie's keynote.)
Ephemerality of the Web is something that needs to be addressed. (Web 2.0/3.0 panel)
One of the ironies of the digitized world is that it's a potential enabler for an unprecedented level of preservation but, in practice, it often ends up opening the door to huge amounts of content vanishing in an instant. At the consumer level, photos offer an illustrative example. The combination of external hard drives and online services can far better protect digital photos from mishaps than is possible with negatives, slides, and prints. But, in practice, most consumers don't have good backup systems and can easily lose everything with the crash of a hard disk. Long-term preservation is an even bigger problem, both online and off. What happens as companies are purchased or go out of business to the content that they create or host?
Web technologies are the best platform for mobile development. (Kevin Lynch, Adobe Systems)
The iPhone hype (or, indeed, the babel that surrounds Apple in general) can be wearing. However, one thing that the iPhone has really accomplished is to crystallize the notion that the browser is a viable interface for mobile phones--at least high-end mobile phones. Kevin's contention that Adobe's AIR runtime is also necessarily part of the mix is more debatable, but the general concept that mobile applications will tend to center around the same mobile technologies that are used on "PCs" seems sound. (I tend to think that the mobile device will be a smartphone rather than a separate "Mobile Internet Device" (MID) but that's a separate debate that centers more around form factors and networks than programming models.)
Many energy solutions are not relevant at the scale that matters. (Vinod Khosla)
Finally, a sizable chunk of the conference was devoted to "green" and energy. Unsurprising given how it's a hot (if also overhyped) topic in IT and elsewhere. It's also an area that lends itself to transformative innovation--which fits well with the general focus of EmTech. Which is what venture capitalist Vinod Khosla, a co-founder of Sun Microsystems, is really talking to with that statement. It's not that incremental changes aren't desirable. They are. Indeed, a lot of power efficiency work in technologies such as microprocessors is a sort of whack-a-mole game of accumulating small wins. However, from a macro and policy perspective, big wins don't come from the niches. They come from making substantial impacts on substantial use cases.
- prev
- 1
- next
