Ask HN: ETL solutions for developers?

by valevkon 7/26/2015, 2:04:34 PM with 9 comments
  • by trekking101on 7/26/2015, 6:27:30 PM

    Ha! There are no general purpose tools--lots are a) overkill or b) not robust enough given your task but two general ones that have served me well (and of course lot of Python scripting for my domain-specific issues):

    Refine (formally from ITA/Google, now open source) IBM Watson Analytics - Don't knock it until you try it. Forget the 'analytics' bit but the ETL bit is pretty decent for getting through the basics

  • by mindcrashon 7/26/2015, 6:41:02 PM

    Data Brewery has a nice ETL library called Bubbles for Python [1]

    Kiba [2] is a really nice library for ETL in Ruby

    Hope this helps!

    [1] http://bubbles.databrewery.org/

    [2] http://www.kiba-etl.org/

  • by uberneoon 7/27/2015, 1:39:18 AM

    If you are looking for commercial solution then try Informatica, otherwise the opensource soln are Pentaho Kettle and Python based http://bubbles.databrewery.org/

  • by Varkiilon 7/27/2015, 11:53:30 AM

    Currently working with Talend, it's pretty good and saved me a lot of time.

  • by mritson 7/26/2015, 8:49:14 PM

    If I was going to start a new ETL project right now I'd pick Luigi.

  • by mathnodeon 7/26/2015, 5:00:49 PM

    Python, and its various libraries including Pandas, and odo.

  • by Aparraon 7/27/2015, 4:19:26 PM

    try pentaho, it's easy and clean, no installation required.

  • by geordeeon 7/26/2015, 4:48:15 PM

    Try Talend