Oracle's lousy job at marketing PL/SQL...
I really think Oracle does a lousy job at marketing PL/SQL to the development community. I've been working here at a small company for just over a year and I still see people cringe a little bit when I tell them I could do something in PL/SQL very easily (and very efficiently and very effectively).
Current case in point: I have a data load I need to take care of and we've decided to take a look at the Pentaho Data Integration tool that used to be called Kettle. This is a pretty cool tool and I'm sure I'll be posting more about it in the future, but for today's rant I've decided to do the load with three modules:
- A Kettle job (build in spoon, run by chef, pretty cool)
- A transformation (build in spoon) that loads a text file into a "staging" table
- A second transformation (build in spoon) that loads the data from the "staging" table into the custom schema
In theory, we would be getting many different data files from many different partners, so when I designed the above my thought was that each partner's "load to staging" transformation would be custom, however the final transformation that loads data into the custom schema would be shared by all partner loads.
So here goes, I figure out how to read the file, apply all the steps (which I think of as widgets) in the visual tool which is really cool and intuitive. After only about ten widgets I've now taken partner X's custom data file and loaded it into our staging table, very nice.
However, building the transformation that loads the data into our custom schema is very very messy. We have lots of rules for this and that and the end result is a transformation that took me a few days to write when doing this in PL/SQL would have taken me probably less then two hours. Now I know that there is a learning curve associated with learning any new software, I get that. But what I'm saying here is that Kettle is a great tool for really easy loads and simple transformations. However, when your Kettle transformation gets to fifty or sixty steps, mama mia it get's messy.
You might ask "How does this make Oracle lousy at marketing PL/SQL"? If there were less grimaces and more acceptance of PL/SQL in the development community (outside the folks who already drink the Oracle punch) I'd have been done with this project a few days ago and I would have used the right tools for each step of the way (Kettle to load the file into the staging table and PL/SQL to do the heavy lifting).
It's not a big deal, at the end of the day I've learned a cool new tool and added it to the toolbox, but had PL/SQL put smiles on everyone's faces this project would have really been great.
Until next time...Rich
Current case in point: I have a data load I need to take care of and we've decided to take a look at the Pentaho Data Integration tool that used to be called Kettle. This is a pretty cool tool and I'm sure I'll be posting more about it in the future, but for today's rant I've decided to do the load with three modules:
- A Kettle job (build in spoon, run by chef, pretty cool)
- A transformation (build in spoon) that loads a text file into a "staging" table
- A second transformation (build in spoon) that loads the data from the "staging" table into the custom schema
In theory, we would be getting many different data files from many different partners, so when I designed the above my thought was that each partner's "load to staging" transformation would be custom, however the final transformation that loads data into the custom schema would be shared by all partner loads.
So here goes, I figure out how to read the file, apply all the steps (which I think of as widgets) in the visual tool which is really cool and intuitive. After only about ten widgets I've now taken partner X's custom data file and loaded it into our staging table, very nice.
However, building the transformation that loads the data into our custom schema is very very messy. We have lots of rules for this and that and the end result is a transformation that took me a few days to write when doing this in PL/SQL would have taken me probably less then two hours. Now I know that there is a learning curve associated with learning any new software, I get that. But what I'm saying here is that Kettle is a great tool for really easy loads and simple transformations. However, when your Kettle transformation gets to fifty or sixty steps, mama mia it get's messy.
You might ask "How does this make Oracle lousy at marketing PL/SQL"? If there were less grimaces and more acceptance of PL/SQL in the development community (outside the folks who already drink the Oracle punch) I'd have been done with this project a few days ago and I would have used the right tools for each step of the way (Kettle to load the file into the staging table and PL/SQL to do the heavy lifting).
It's not a big deal, at the end of the day I've learned a cool new tool and added it to the toolbox, but had PL/SQL put smiles on everyone's faces this project would have really been great.
Until next time...Rich
Comments