What would a cloud-based ETL tool look like?

re: What would a cloud-based ETL tool look like?

Joe Harris — Tue, 12 Feb 2013 15:03:58 GMT

Funny you should mention it but I was just reading up on this yesterday.]

Here are the 2 best articles I've seen:

1) http://www.apievangelist.com/2013/02/10/bringing-etl-to-the-masses-with-apis/

TL:DR - It needs to be an API focused version of Yahoo Pipes.

2) http://seabourneinc.com/2013/02/08/rethinking-etl-for-the-api-age/

TL:DR - ETL needs to become AP2 (Acquiring, Processing and Publishing)

Personally, I like want something totally different. I want an API focused tool that provides a nice syntax so I can script these actions without the tedious "drag > drop > click > fill_form > repeat" of current tools.

Joe

re: What would a cloud-based ETL tool look like?

jamiet — Tue, 12 Feb 2013 15:14:51 GMT

Hiya Joe,

Sounds like you want something like Datasift (http://sqlblog.com/blogs/jamie_thomson/archive/2010/12/17/archiving-sqlhelp-tweets.aspx). Its got what seems like a very nice little DSL (http://martinfowler.com/dsl.html) for doing ETL on APIs.

I like AP2. Nothing is real util its got a TLA :)

re: What would a cloud-based ETL tool look like?

Chris Nelson — Tue, 12 Feb 2013 17:23:10 GMT

I took a look at some of the examples and they seem to be more news/text aggregators than true data driven APIs. A quick look at MyFCC doesn't appear to highlight some of the core open data available from the FCC ULS database. (It may, but it's not obvious at a glance.) There's huge amount of open data out there, but much of it requires someone familar with the domain to make it useful.

So instead of dealing with thousands of different file formats, we can deal with thousand of new API's?

re: What would a cloud-based ETL tool look like?

jamiet — Tue, 12 Feb 2013 17:29:36 GMT

"So instead of dealing with thousands of different file formats, we can deal with thousand of new API's?"

Of course, for every problem you solve with an extra layer of abstraction you simply create another problem :)

re: What would a cloud-based ETL tool look like?

Phil — Thu, 14 Feb 2013 18:42:15 GMT

Dell Boomi?

ETL is dead, long live AP2 ?

SSIS Junkie — Fri, 15 Feb 2013 15:45:34 GMT

Three days ago I posted What would a cloud-based ETL tool look like? where I wondered out loud about

re: What would a cloud-based ETL tool look like?

Kent B — Fri, 15 Mar 2013 21:00:03 GMT

Have you looked at Informatica's offerings?

http://seroter.wordpress.com/2012/03/26/etl-in-the-cloud-with-informatica-part-1-sending-file-data-to-dynamics-crm-online/