The problem of searching for media assets.

We’ve already looked at advanced media, in particular video, audio, and animation data, in previous blog postings. In particular, we’ve looked at the subtle and complex nature of media asset semantics. We’ve seen that interpreting a piece of video, for example, is far, far more difficult than interpreting an integer or character field. Since the goal of the Semantic Web effort is to make the searching of the web highly automated, advanced media is becoming a huge and critical research and development focus for the builders of next-generation web development applications.

Just how do we provide an environment where media assets can be searched in a mostly automatic fashion, so that a human does not have to painfully paw through hundreds or thousands (or millions) of video chunks to find the right one? We’ve looked at emerging technologies for marking up advanced media information, and for making it usable in a variety of web applications. We’ve also looked at the dramatic challenge presented by mega apps to would-be users; the interfaces to these applications are truly massive and cannot present to the user the way in which they are meant to be used.

The problem of proprietary formats.

One specific, and very difficult problem, is the massive heterogeneity, not just of media formats, compression technologies, and container technologies, but of the applications themselves. If we are going to automate the searching of complex modeling, video, audio, and other media assets, we’re going to have to address a key question: since many media apps make use of their own proprietary data formats, how are we going to provide automated ways of searching media assets that are stored in these formats?

The problem of highly imperfect generic formats.

There are indeed many existing, as well as soon-to-emerge, standards for importing and exporting data between powerful media applications, but transformations in and out of these formats are often “lossy”, in that information is lost or changed. In fact, locating and downloading assets that are in supposedly-generic form is often very frustrating, because these assets end up not performing well. They can be difficult to edit and reuse. 3D animation models regularly blow up when animators try to import them into animation applications and the manipulate them. A hawk may look like a hawk until you try to render it with its wings flapping, and suddenly it’s a blob of geometric garbage.

One possible direction.

So, what do we do about the fact that many media assets must be manipulated by the original applications that created them? How can we facilitate reuse? It’s extremely unrealistic to expect users to master perhaps dozens of video or audio or animation applications. Filtering assets according to their file extensions is a good idea, and it is a well established practice.

But what we really need is a globally-known site that either literally or conceptually centralizes the massive network of import/export relationships, along with information about the relative success of these mappings. Are they ever lossy? If so, can they be fixed? What series of applications might we want an asset to be imported/exported through so that in the end it is in a usable format, given the applications that the user owns and has mastered?

There is much to be done. Right now, searching for and reusing media assets is a painstaking, trial-and-error-prone process.

Let’s start with the definition of a third term. A Web Application is a website that provides some sort of substantive functionality other than simply filtering and presenting information. Evernote is a fantastic web app that stores your notes on a server, and allows you to create, group, and annotate your notes. Some folks say that a web app makes it clear that there is an application at the other end of your browser, and not just a bunch of static data. This is admittedly a pretty soft definition, but it’s reasonable. Another way to look at it is that a web app provides what would otherwise be a desktop application, but makes it accessible from a server so that users do not have to install and maintain an application.

So what’s Web 2.0? It refers to web development frameworks and tools that can be used to create highly responsive websites and web applications. AJAX does this, and the conical example people give is Google Maps. AJAX allows data to be retrieved asynchronously while a prior page is being displayed and manipulated by a user, and minimizes the amount of a web page that must be replaced with the next refresh.

A somewhat newer approach is embodied in Adobe Flex and Microsoft Silverlight technologies; in these cases, a web app is sped up by running more of the application’s logic inside a browser plugin (Adobe Flash or Microsoft Silverlight), rather than making the client machine (which runs the user’s browser) continuously talk to the web server. The overall challenge is to make web pages highly dynamic (meaning the data comes from a database and is not hard-coded in the web page) while giving the user response times that approach those of a desktop application running on a dedicated or near-dedicated machine. While this is intractable at this point, it’s a good thing to hold up as a goal.

The term Semantic Web does not narrowly refer to technology that speeds up response rates. Rather, it refers to a still emerging body of software tools whose overall goal is to automate the collection and integration of information gleaned from websites. The idea is to free the Google/Yahoo user from painfully interactive, highly repetitive keyword searches where we continue to hone our queries until we seem to be finding the right stuff.

Semantic Web technology includes namespaces, which try to put more smarts in websites by having data tagged with widely shared, standardized sets of tags. And things like XML Schema and XQuery can be employed to leverage namespace technology to support high-volume, set-oriented queries of data stored on web servers. These are very similar to the sorts of queries that can today be coded in SQL and run on single database servers running database management systems like Oracle, SQL Server, DB2, MySQL, and PostgreSQL. Essentially, XML-based technology takes the ability of a relational database schema to help us interpret data, and extends it to the entire web.

We will look at XQuery and XML Schema in future entries of this blog.

By the way, some folks are already talking about Web 3.0, which in many ways draws from both Web 2.0 and Semantic Web technology. We’ll look at this in a future blog, but a key focus is on making web apps highly multimedia.