Howto visually design a mashup query for programatic extraction

Question

I'm into development of an application that fetches various inputs from internet pages whereas each information snippet comes from a different location (mashup). I would like to generate the mashup building block (snippets) through a visual tool. Do you know of anything similar that can be used for such a project? (Already made control, a sample code, article, etc.) Preferred development environment is .NET - but not mandatory.

Answer 1

IMO the major challenge will be to extract the appropriate information from each feed in semantic form. Wikipedia describes mashups as:

There are many types of mashups, such as consumer mashups, data mashups, and enterprise mashups. The most common type of mashup is the consumer mashup, aimed at the general public.

Data mashups combine similar types of media and information from multiple sources into a single representation. One example is AlertMap, which combines data from over 200 sources related to severe weather conditions, biohazard threats, and seismic information, and displays them on a map of the world; another is Chicago Crime Map, which indicates the crime rate and location of crime in Chicago.

The classic mashup - Chicago crime - works because key information such as dates and geolocations are available semantically. Other types of common information are persons, organisations, and domain-specific identifiers.

When you have identified these you may wish to consider the RDF-based tools that the semantic web is developing. Note that governments are starting to emit their data in RDF so I would see this as a key technology

If your web pages do not have semantic information immediately you will probably have to create screen scrapers and HTML parsers. That's not very glamorous, there are no special tools and tends to be just hard work.