General anxiety for the BI/DW developer is how to cope up with changing technologies and learn something new to move on.
However, if you understand the evolution of technology its always learning new things and adapting to the change. Someone who wants to embrace the change then he has to put the required hours of dedication to learn the technology.
The learning new things always uncomfortable until you find the chord which brings music nice to hear. We have to put the required efforts to get the music going on or find the right instrument which suited for our skill.
I have listed down a few of the general things DW/BI developer should embrace to become a big data engineer.
Traditional BI/DW developer keeps asking a question - How can I survive the big data wave and still be relevant in the industry. There are n number of technologies and buzz words makes any developer uncomfortable. The issue here is many customers or peers in the industry have borrowed the words from some webinars, blogs, interaction with coworkers.
Word Big Data is very scary in the mind of traditional data warehouse developers. I was one among them a few years ago. Not any more, big data, small data, stream data any kind of data its end of the day data to be managed, processed, analysed and presented for consumption. I would suggest that slowly our industry will normalize and start looking for the talents which talk only about the data engineers irrespective of the amount of data he has to manage. The tools and technologies used to manage the different volume of the data are different which has created the separate league of the data management folks called Big Data engineers.
Here are a few tips I pen down, these ideas will help traditional DW developer to become a big data developer or become relevant in the world of big data. These are few open suggestions, not hardcore list to be followed and I am kind still working on the transition path.
1.Books
2.Blog
3.Webinars
4.Programming
5.Project exposure
6.Cloud Solution
Learn some programming language such as Python, Java or Scala. This is mandatory for anyone willing to jump into the big data world as a developer. It might sound little difficult to learn programming initially however it will help a lot to move forward in that direction. Most of the technologies use the programming for writing the map-reduce function or data processing function with few built-in libraries from the base language. It could be programs written using python or Scala or Java
Books -
Read at least 3-4 books on the Big data subject to understand varies jargon of the industry. There are terms which might be new for the people who are moving from DW to Big Data world. Get familiarized on those terminologies.
Start with simple books. Don't try to buy all the books, probably you can try to borrow the books from the library for reading and explore the options initially.
Blog -
Regularly read some blogs related to the big data world and try to contribute to the blogging community by putting your efforts. Understand the pattern and trends used to solve the big data problem. Try to document the learnings into words or teach someone what you have learnt. This helps in 2 ways, one consolidates the learning which you have achieved and second inspires your coworker to start working on learning the subject along with you.
Webinars -
Attend at least 2 webinars in a month to get to understand what is the trend in the industry and what new technologies being released. Webinars provide greater detail of understanding which you get instantly instead you might need to spend 3-4 hrs to know a few simple new things.
Ask questions in the webinar to solidify your understanding, read some relevant posts, technical material related to the webinar, follow the webinar series to get the updates.
Programming:-
Let me explain, how I am learning to programing and still learning it.
I started learning python, here are a few things which I started to work on it.
1. Pick up some online course of Python there are many courses available online for free in Coursera, edx, udemy etc.
2.Complete the course with all exercises and project work
3. Buy any book on python, I am reading "Learn Python the Hard Way" by Shaw. This book is very good and teaches programming for the layman. It explains the concept from basics and builds the knowledge of the subject slowly.
4. Once you complete the book "Learn Python the Hard Way" then go and buy the "Learn more python the Hard Way" by Shaw next series in the same publisher
5. Subscribe to Stackoverflow channel and check for the problems which you can solve.
6. Work on the open-source projects so that you can help community developers
Repeat the steps 3 to 6 as long as you are master of the subject. This is a time-consuming activity but only way to master the subject.
Project Exposure -
Start working on the project which applies the techniques learnt during the process. Don't try to be over-engineer the project with all your learning, apply the simple techniques and explore the way forward. Make sure you are part of all phases of the project and implement solutions. Put your hands and mind into places where you feel uncomfortable. Things which will put you on the right path are things which make you uncomfortable.
Cloud Solution -
Pick any cloud solution and explore the solution implementation using the stack. It might take some time to absorb the concepts of the cloud with respect to on-premise licensed solution implementation.
AWS, Microsoft, Google cloud providers are leading solution providers who are engaged in the great transformation on the cloud solution implementation.
Conclusion - learning or mastering big data engineering is not rocket science. Anybody willing to put the required effort and learn things will master the techniques in no time.
Go out play it fully and become a Big Data Engineer. All the Best
