AI and ML Meets DevOps – Part V

What challenges are ML Engineering team facing? 

Key success factor that defines a ML Engineering team is that they have access to Data and then they have automated their software engineering to achieve agility and efficiency in delivering value to the business. A ML Engineering team faces following challenges in two broad category.

  1. DataOps – Data Engineering
    1. Data collection from different sources and extraction 
    1. Data pipelines that primes for analysis and models
  2. Data Science 
    1. Develop features and models
    1. Train Models
    1. Discover new features
  3. Data: Not able to get the production data “quickly” for analysis and feature engineering

These challenges make getting machine learning (ML) into production a difficult and painful process. In comparison with classical software development and deployment, AI/ML engineering and deployment is an order of magnitude harder. As a result of this challenges, most AI/ML models never see the light of production-day. Thus, due these challenges, many organizations fail to capture benefits of ML quickly to capitalize on their data and provide their customer with next generation experiences.