Learning Path for Developers & IT Professionals to become a Data Scientist

Kunal Jain 26 May, 2016
9 min read

Introduction

This guide to meant to help web developers, software engineers and other IT industry people to transition into analytics / data science industry.

Last week, I was taking a guest lecture with one of the well known institutes in India. Rather (un)surprisingly, more than 60% of the students comprised of experienced IT Professionals. Most of them are facing a common problem, “I have been in IT / software / web development for more than a few years and want to up-skill myself in analytics. I have taken a few MOOCs and have tried using a few books / platforms. Still, I don’t get it what should I do next?”

This scenario is not very different from several learnups / meetups we have conducted over the last year. In order to help these people as much as I can, I’ve created this comprehensive career guide to get you quickly started with data science. Once you finish reading this post, you would know the next steps in making a transition.

webdevelopertodatascientist

 

Self-assessment: A summary of where you stand

People working in IT industry are generally comfortable with coding, working with databases and using frameworks. After spending a few years as a developer, you would know at least some languages like Java, ASP.net, Javascript, C++, C, HTML, Python, PHP, and you would have worked with several databases including SQL, MongoDb, Oracle etc. With these skills, you are envied by people trying to transition from non-programming background. Ask them, how they feel about it!

far away from goalLet me explain the set of skills (eventually advantages) which I expect every good IT professional to have:

  1. Coding Experience: If you are comfortable with programming, you can focus on understanding the concepts and solving the problems. This can be a huge benefit in the initial days. Even though you might not have experience in languages used in data science, you still have an advantage as you understand the concepts like indexes, functions, objects, referencing etc.
  2. Logical Thinking: Data Scientists are logical. They make data based decisions. So, would you! Coding requires logic. Be it the programming logic. Your brain is accustomed to think that way. Definitely a plus point for you!
  3. Numerical Ability: Mathematics forms the core of data science. To be good with numbers is the biggest achievement one could get. I’ve seen that people who are good at coding are also good with numbers.
  4. Database Knowledge: IT professionals are known to have good knowledge of database. This knowledge is immensely helpful for data scientist to quickly obtain data from database teams.

On the other hand, you can be fooled to believe that data science is about learning a few more tools. Just like in programming, knowing a few languages or frameworks would not make you a good software engineer. What differentiates a good analyst from a bad one is the problem solving and structured thinking skills. Tools are just a way to implement this thinking. Hence, I recommend people to just pick one tool depending on their convenience and then focus on getting hands on experience.

Here are the areas you need to focus on going forward:

  1. Structured Thinking: Structured thinking refers to the art of taking an ambiguous problem and then putting a structure around it. Remember those days, when you got a very loosely / generically written requirements document from a client? This is bound to happen often in analytics. Your clients and stakeholders will tell you the business problem and you will be expected to put a structure around it and solve it using data science.
  2. Math & Statistics: You should know some basics of statistics, algebra and numerical inferences to start with. You can enrich this further over time.
  3. Domain knowledge: As a data scientist, you would need to know the domain very closely. Why is a particular process being done that way? If you are in banking, you need to understand why risk management is important and what are some of the common terms used in the industry. You need to spend some time understanding the domain and current processes.
  4. Presentation skills: As a developer, you are mostly good as long as you can explain your code to fellow developers. Not all developers need to co-ordinate with the client and present their solutions to the client directly. On the other hand, a data scientist / analyst would need to present his / her findings in a manner which is easy and exciting for the business.
  5. Technical Data Science skills: Starting from ways and biases which can arise in data collection, cleaning to the predictive modeling and implementation of data science solutions – you will need to pick all of it in coming days.

 

Where should you start ?

The problem in today’s world is the problem of plenty. I am sure you couldn’t agree more. Try searching for resources on Internet for R / Python / Data Science and you end up with a long list of resources. Talk to a few people who have made the transition and they would add a few more resources, which worked for them. If you are an avid reader, you can add a few books and blogs over and above this. Check out the platforms offering MOOCs and you can see a few good courses.

The sad part is that while you have access to plenty of resources today, you find it difficult to find your way through these resources. Hence, I have created this learning path:

 

Step 1: Getting your machine ready

The Hardware:

Data Science is computationally intense (this should not be a news to you!). So, the first thing you should do is to set up a machine which helps you in your learning. Ideally, I would say that any machine used for serious data science work should have at least 8 GB RAM and an equivalent of i5 / i7 intel chips. Of course, the higher capacity you buy, the better it is! If you have some more money to spend at this stage, you can even get a SSD upgrade.

The Operating System:

Ironically, there is no single OS which works perfectly for Data Science. You would likely need a mix of Unix and Windows machine. Unix is better in resource management and for performing the data science work. On the other hand, you would need Windows for Powerpoint and Excel, both of which are used very heavily in data science work flows.

Also, there will be a few visualization tools, which work better in Windows environment. Hence, I would recommend to use Linux as the core OS with a virtual machine running Windows or vice versa. If you are used to Mac and can work comfortably on Excel on Mac, you might be good too.

The Softwares:

You would need to choose the language / tool of your preference here. If you have experience in coding with Object oriented languages, I would recommend Python. It is easy of learn and has a vibrant community on internet. If you aren’t used to object oriented programming, you can give R a try as well. If you need more details before making a decision, read this comparison – SAS vs. R vs. Python

Here is a list of softwares, I would recommends at the minimum:

  1. MS Office for Excel, building presentations and writing documents.
  2. FileZilla for transferring files using FTP
  3. Git & GitHub for version management.
  4. VMWare / Oracle Virtual Box / Vagrant for running virtual machines
  5. Cygwin / Putty (for windows)
  6. I use Evernote for taking notes. In case of Linux, you need to run it in the browser.
  7. Terminator (for Linux) to run multiple terminals in a single view (it is awesome!)
  8. Sublime Text (or any other editor of your choice) for editing codes. You should install additional plugins for the languages you use.

If you have chosen Python as your preferred tool:

  • Install Python (2.7 or 3.4) with numpy, scipy, matplotlib, scikit-learn, pandas
  • Install Jupyter notebooks
  • In addition, Dato (Graphlab), vowpal-wabbit, import.io are some interesting additional libraries

If you have chosen R as your preferred tool:

  • RStudio is a good choice of environment.
  • Install important packages like dplyr, lubridate, ggplot, caret, randomForest etc.

Your machine is ready to crunch some numbers now!

 

Step 2: Getting used to solving ambiguous problems (i.e. Developing Structured Thinking)

The art of structure thinking is a tacit requirement but profoundly being sought in every data scientist. Otherwise, below are some good resources to enhance your skill.

Assignment: Solve this case study on operational analytics: Call Center Optimization. It’s a beginner level assignment. Once you complete it successfully, you can move to medium and advanced level.

Think you are ready for the next level, try out our practice problem on Strategic thinking

 

Step 3: Basics of Math & Statistics

Mathematics plays an important role in defining data science. Thankfully, you don’t need to learn all of math, just a few topics would do. You can start from the basic topics (marked mandatory) and pick up the rest of the topics as you progress:

Assignment: Do a statistical analysis on Big Mart Practice Problem. After you have finished with this assignment, you can showcase it on your LinkedIn profile as project work.

 

Step 4: Basics of new tools (R & Python)

After step 3, you should do programming. Coding in data science is laconic in nature. Best practitioners avoid redundant lines of code and adopt ways to make it faster. Your prior knowledge of programming basics, should give you a nice head start in solving practical problems using R or Python.

Get used to the basics of R / Python from any of the following introductory courses:

Apart from these 2 tools, you can also use Julia, Go, Java to build predictive models. However, a possible drawback with other programming languages is the lack of community support. Till now, Python and R have the best community support on web. Thus, would help you to debug issues and learn faster.

Assignment: Already given the practice links above.

 

Step 5: Get your hands dirty – Your First Project

Time to get your hands on your first project. Like programming, the best way to learn data science is to do data science. Hence, let us start by taking up a problem to work on. You can choose any of our Practice problems or any of the projects mentioned here to start with. Perform an exploratory analysis on the data to get you started.

Here are a few good places to look at:

Assignment: Perform a similar analysis on the project of your choice.

 

Step 6: Follow the steps in learning path

Now that you have tasted blood, go for the kill! Check out our learning paths on R & Python – follow them step by step. Skip the steps you are comfortable with. Do as many exercises as you can!

Here are some additional machine learning resources you can look at. Remember that the best way to do data science is to learn data science thoroughly:

Assignments: Build a machine learning model on Loan Prediction Problem using the following algorithms:

  • Logistic Regression
  • Decision Tree
  • Random Forest

For Decision Tree and Random Forest, you can seek help from: Complete Guide on Tree Based Modeling.

Make sure you understand how each of these algorithms works. Just implementing them and obtaining predictions wouldn’t be a success for you. The real success lies in gaining knowledge of how they work!

 

Step 7: Start participating in the competitions

Time to step in battle ground. A benefit of being a part of analytics community is that you get to access so many thrilling ways of learning concepts. You no longer need to stick to traditional ways of learning.

Competitions: Several data science competitions get organized across the globe where you can participate and win prizes too. After you’ve completed above steps you must participate in these competitions to assess your learning level.

Assignment: In 6 months – 1 years, try to rank in top 100 in the competition rankings on both websites. This would give a massive boost to your profile.

 

Mistakes You Should Avoid While Learning

We are prone to make mistakes (unknowingly) in pursuit of learning concepts quickly. Nothing to worry, we all are susceptible to such things. But, we need to be prudent enough to analyze our learning pace and proceed accordingly. Below are the list of mistakes ( bad practices) you should avoid while completing this learning path:

  1. Choosing difficult problems: Working on difficult problem during initial days wouldn’t make you a kick ass data scientist. You’d only end up torturing yourself. Follow a natural pace. Start with simple problems as given in assignments above. It would establish your much needed confidence.
  2. Learning too many concepts: You might get overwhelmed with the availability of online resources. Don’t get swayed away by their abundance. You don’t have to be a “jack of all trades and master of none”. Become a master. Take one tool or one concept at a time. Once you become confident about it, move to next.
  3. Losing way in coding:  You might have done die hard coding in last few years. But, data science is a bit different. You are no longer writing softwares. You are using them here. Don’t waste time in writing algorithms. Use Libraries (or packages). Tools like R and Python enjoy an incredible support of powerful libraries and packages. Use them. In short, figure out ways to do more in less lines of code.
  4. Getting Impatient: This is the most important of all. I’ve met people who’ve claimed to spend hours on learning data science. Yet, they haven’t got a break in this industry. You need to stay patient. After all, good things take time. You’ve got access to amazing community support @ Discuss. Engage more. Share your queries. Let people know what you are up to. Remember, you are not alone with this, you are with us.

 

Finally, a peek into the Life of Data Scientists at Companies

Just for some motivation…

https://www.youtube.com/watch?v=7mXFkvu98vI

 

End Notes

I hope this guide will help people from IT / software development background to take up data science / machine learning as a career option. In summary, rely on your strengths, focus on developing structured thinking & problem solving, practice a lot and get your hands dirty on as many real life problems as possible. In the process, if you get stuck, leverage the communities and people in your network to help you out.

As usual, if you have any questions or suggestions I might have missed out on, feel free to reach out to us through the comments below.

You can test your skills and knowledge. Check out Live Competitions and compete with best Data Scientists from all over the world.

Kunal Jain 26 May, 2016

This is a test bio profile and should be working well. I have worked at X, Y and Z across different markets. Have been building AV for the last decade. Still needs more characters! jkhdjkfhd kjdhf kjhdakj fjfh oidsh fjfhdsjkh fkjdh fjkdh fkjdj fdkjh jk dsfkj dkjfd fjkd kjf dkjsf kdjh jdjkf dskjf dj dshf jdkh fkjlds f

Frequently Asked Questions

Lorem ipsum dolor sit amet, consectetur adipiscing elit,

Responses From Readers

Clear

Vishwachandra
Vishwachandra 26 May, 2016

Kunal, The effort that you are making to help the data ppl is very commendable.Please keep it up.

Venkat
Venkat 26 May, 2016

Hi Kunal, Great stuff!! I am sure many aspiring data scientists will find this useful. Thank you for your help. Venkat

Akshatha
Akshatha 26 May, 2016

Thanks a lot, Kunal. This is very useful to me. I'm planning to switch my career to Analytics and I hope I will be successful soon by adopting the above mentioned steps.

Ashok
Ashok 26 May, 2016

Awesome post, Kunal

pradeep
pradeep 27 May, 2016

Good one for starters...

Moeen
Moeen 27 May, 2016

I am one of the IT professionals who has transitioned to analytic and big data area, the only point i would want to mention is, keep doing what is been mentioned by Kunal and over time you would get an opportunity. It took me nearly 1.5 year to get in analytics field.

Srinivas G Rao
Srinivas G Rao 27 May, 2016

Hi Kunal, This is really a great article that I have ever found. I wish every aspiring data scientist (even myself) to make use of it to achieve their goal. Many thanks to Analytics Vidhya in providing the base/foundation for this type of articles from this great authors. Regards, Srinivas G Rao

Rambabu
Rambabu 27 May, 2016

It is very nice Article, Based on the article, one can know what is the pre-requisites to learn data science

ksmvsn
ksmvsn 28 May, 2016

Excellent work Kunal. This is what i was looking for, since i am IT professional looking to change from IT to Data Science.

Ragini
Ragini 28 May, 2016

Any Recommendation/Learning path for Statistics lecturers wanting to move into Analytics/Data Science I am a Stats lecturer with 1 year of experience in teaching to undergraduates

THANIKACHALAM
THANIKACHALAM 28 May, 2016

Super

GN
GN 28 May, 2016

Hi Kunal, I am IT professional with 10 yrs of exp in testing n BA in finance. I am looking for a course which will help to switch a career as data scientiest. 1 great lakes pgpba 2 ms program by ibm n aegis 3 pgdda by IIT banglore n Upgrad Which one is good?

srikanth
srikanth 29 May, 2016

i required guidance on which institute course will be better practical real time oriented for me to start with in data analytics im non it background. plz suggest some good institute to join

Jayashree
Jayashree 30 May, 2016

I am a Java Professional with 3 yrs experience.I have taken a break in my career due to child care.I want to make a career as data scientist. 1.Most of the Jobs posted in LinkedIn requires data scientist with 5+ yrs experience. What about the requirement for someone who is new to the field? 2.Does the short term certification courses provided by online training academies is on par with PGDM in Business Analytics provided by B-schools?Are these certifications industry recognized? 3..As data scientist is an umbrella term and there are many roles around it like data visualization expert,machine learning expert. Pls list down other roles as well. Thanks in advance for taking time to answer these questions.

Manoj Kandel
Manoj Kandel 30 May, 2016

i am new learner so do you have any basic tips for candidate like me? please post it.

Manoj Kandel
Manoj Kandel 30 May, 2016

please post the basic analytical tools for the fresh learner.

Olumide Michael Oyalola
Olumide Michael Oyalola 30 May, 2016

Very insightful. Thanks for sharing!

Naresh Buch
Naresh Buch 31 May, 2016

Great Stuff . Very much helpful to get into the course

Manish Jain
Manish Jain 02 Jun, 2016

Really nice points to start with transition to Data analytics. Sincerely thanks to you Kunal for providing really good guidance to the interested ones. It really helps. Kepe up the good work and let the community grow each day.

Mounir
Mounir 02 Jun, 2016

Hello; very interesting article. thank you for the advice, Can you give an estimation the time required to complete a Step before going to the next one ? I know that it depends on the persons skills. but let say for someone who have (My profile) - not much experience in programming (I used to program in C; Delphi in the university and I was one of the first in class and I learn C++ by my slef but I didn't work on any big project. - good skill in math and analytic thinking to resolve problem Thank you for answer in advance,

Marta Seoane
Marta Seoane 02 Jun, 2016

What an excellent guide, Kunal. Thank you for the contents and efforts.

amit
amit 08 Jun, 2016

hi, i am mechanical engineer working in oil and gas Engineering management profile. can i go for the business analytics?? As i want to switch from oil & gas industry

Ammar Gaber
Ammar Gaber 08 Jun, 2016

This is the best guideline for data scientist I ever found. Thanks Kunal and well done!

Rohit Patil
Rohit Patil 09 Jun, 2016

Thanks so much Kunal for such a beautiful explaination. This is the first article of my life which I read from start to end with interestingly. ?

Rohit Patil
Rohit Patil 09 Jun, 2016

Oops...missed to mention that I have bookmarked it and will start following it...?

Robert
Robert 15 Jun, 2016

Hi Kunal, Excellent guide to switch over to data science. I am 15+ yrs experience in Oracle technologies mostly application programming and database design. Is the correct choice for me to move to data science or Big data development or not ? . Can you and all give your feedback and i am looking this transfer both for career growth , sustainability in IT industry from next 10 yrs and better finance.. Regards Robert

Rabiya
Rabiya 25 Feb, 2022

thank you for the advice, Can you give an estimation the time required to complete a Step before going to the next one ? I know that it depends on the persons skills. but let say for someone who have (My profile) - not much experience in programming (I used to program in C; Delphi in the university and I was one of the first in class and I learn C++ by my slef but I didn't work on any big project. - good skill in math and analytic thinking to resolve problem