Exposing your data!

Open source and open standards are a major thing. For some, it’s the antithesis to commercialism and modern capitalism. But I think that’s quite a shallow and biased view. In the media I’ve caught glimpse of placing Europe as the old, industrialist nation that wants to protect its citizens from unfair competition by slapping US companies with hefty fines. I don’t know if this can fairly be said. Law is very abstract for me; especially that of continent-wide competition and international issues.

Internet Explorer’s place in Windows can well be justified. If I said that Internet was merely an obscure backyard that people would seldom visit, I’d be red in my face. Of course you want to surf the net. Only the most recluse writer or professor could stay out of the global network of thought. The decision to withhold IE from Windows 7 feels like an ATM vendor would be obliged to only supply the hardware but let each person choose their own user interface for the money withdrawal.

But let’s get to the point. I’m writing about open source and service components, and
the possibility that the services would have open interfaces. Facebook, Twitter and
many other new media services are exposing their data, intentionally. These tools
increase their usefulness by letting other developers take benefit of existing
technology. People can write client programs to make Twitter look different, in
radical ways.

What we have currently is a highly evolved world wide web. There’s pictures, animation, sounds, videos, text, functionality; complete programs, etc. A lot of things, that is. You can start to really live your life online. Doing spread sheet, word processing, blogging, messaging – it’s possible on WWW. So there’s an immense
amount of data, both well-organized and chaotic. But what we don’t have are open interfaces (APIs) to this data. They are actually increasing in amount, but we’re still infants on this area. Google has a good
supply of open interfaces; so do the aforementioned messaging/social services.

What’s often available is a certain service, like a bus schedule site. It’s evolved over time, and contains everything you could think of. But you can’t use the site
programmatically – through another piece of code. In other words the site doesn’t expose its database to any other user. Imagine how neat it would be to be
able to scrape the schedules, and use them in connection with say, a calendar software. If you received an email from a friend, and it contained an identifiable place+time combination (“let’s meet in W 42nd St and 7th Avenue, New York, on 21st June”), your personal assistant program would instantly lookup possible methods to getting there, plus the weather! You wouldn’t have to navigate in a web browser, search the screen, click on several selections, and then get the results. No, you’d instantly get several useful pieces of information regarding the meeting.

There are some desperate attempts to “rob” information from sites. It’s called scraping the web. But this is quite a heavy method, and if the site presents its data in another way, the scraping program has to be accommodated in accordance. So it means constantly watching for changes in the source web page and staying abreast.

Databases are at the heart of business logic. They feed the systems that work on the data. So that’s why it’s so easy to do the decision to lock down access, and only offer results derived from the base data. It’s the “traditional” point of view to business. The new wave is to make components, which can be used by other developers. Of course as we’re talking about entrepreneurs and companies, there still has to be
some business logic in this behavior – it has to be justified and rational.

Exposing data has one more major thing: when you do that, developers will surely
start to bombard you with questions, reporting inconsistencies, etc. This can be
very demanding on the company. No matter what kind of AS-IS statements you make,
you still get a lot of communications. Wise companies take benefit of this
instant and free feedback, while old-school companies possibly even withdraw their
offer and seek asylum.

When I used to work for a map-making company, I was quite horrified by the license costs of map data. Individual people would often like to use a piece of map, just for a passing opportunity – advise friends how to drive to their birthday party, or show where their cabin is. Legally this requires a license. So the response on people’s behalf was to draw maps themselves, scan telephone directory maps, and what not. I mean it was really creative how the licensing was avoided.

But, that’s not true anymore. Google Maps has made it possible to link to a clear piece of map, and not have to pay any license fees. And I think it’s great! But it brings to front the question, what kind of information should be made free? And can governments play a part in this? Should they? What is Google suddenly decided to change the terms of service? Let’s say 50 million people had made homepages with a map link, and suddenly the terms changed and said it wasn’t legal anymore. Even though I don’t believe this to be likely, it’s possible. Do companies have some liabilities about the terms they provide, or is it ok to change them arbitrarily?

I have to say there are more questions than answers in this article. If you
know more about the legal issues, especially regarding terms of service and
the process in how they can be changed, feel free to contact or leave a comment
here! Thanks.

Leave a Reply