Daniel Blum

How to Leverage a Machine Learning Solution Before We Start From Facebook feed recommendations to traffic alerts on Google Maps – machine learning (ML) and artificial intelligence (AI) are (everywhere/ubiquitous); even in the news. AI is a trendy term that many journalists use (in connection with / to describe) emerging technologies. While often used interchangeably, artificial intelligence and machine learning are not the same things. Let’s explore these terms and gain an understanding of both. What Is Machine Learning? Before delving deeply into machine learning, Let us consider the difference between two over-hyped terms; AI and ML. AI is a term used to describe software that can solve problems by itself, without human intervention. Machine learning is a subset of AI that allows computers to self-learn, over time, without direct supervision. While it is indeed true that ML is a promising technology, it’s not a silver bullet for all problems. In short, machine learning consists of algorithms that “eat” labeled data, look for patterns, learn from them, and then make predictions. But wait! AI and ML go hand-in-hand. Why is this? The reason is that AI uses many ML algorithms as tools (we’ll cover them shortly) as the best way to find a solution. So, when a person talks about AI / ML, 99% of the time they must talk about both. To get a better understanding of the big picture, let’s look at an NVIDIA explanation of the topic: Deep_Learning_Icons_R5_PNG.jpg-672x427.png Source – NVIDIA Blogs Machine learning involves finding patterns in structured data and making sense of it. It differs from traditional software systems in the sense that it figures out answers dependently, using the data chunks we feed it, instead of an engineer programming the system as to what to do. Google’s Chief Scientist (name?), decision-maker says there’s an analogy that might help us understand what ML is. It’s called the fancy labeling machine: After teaching a machine to label things based on images of fruits, the machine, after some time, can label other images of fruit correctly without any outside help. An example is shown and the system learns it and labels it appropriately. How Machine Learning Works [img]  1. The algorithm “eats” input data images and a set of appropriate tags. 2. The input data is transformed into an array of numbers that represent different features of the images. 3. Based on data samples that have been pre-tagged manually, and working arrays, the algorithm makes predictions while processing “unseen” data. (Never before seen? /data never seen before?) (I need clarification about this last point before I can be sure how to write it. Maybe I need to see the interactive examples.) Interactive Examples See how the R2D3 team built an animated visualization of a machine learning algorithm.  You also can tinker with an example of a neural network in your browser to see how it learns.  Watch these ML experiments page on GitHub. Most Used Machine Learning Algorithms [table img] Algorithm Supervised Learning Unsupervised Learning Training Data Type Manually Labeled Unlabeled / Circumstantial? (Is this the right word?) Desired Output Clear Unknown Use Case Spam detection, pandemic-spread predictions Marketing customers segmentation, demand forecasting What to Choose? If the desired results an algorithm is to produce are known, supervised learning is used. The model will learn from data samples that have been pre-tagged manually from their source. Using this approach, complex AI/ML solutions can be built to better understand customer behavior and improve their experience. It should be considered that it’s not about pulling a lot of data to solve a certain domain problem. Keep in mind the scalability of the solution, narrow down the goal as much as possible.  If what is expected from the datasets is unsure, unsupervised learning is adopted and the algorithm is allowed to produce outputs of its own. The unsupervised model is useful for analyzing “raw” data and clustering datasets. It can be used to identify customer behaviors or groups to deliver a tailored solution for every client. An ML algorithm will give what is asked for, not necessarily what is wanted. That’s why it’s crucial to be thoughtful about what is asked for. Machine Learning Infrastructure [img] Data: collecting, verifying, analyzing, preparation Infrastructure: deployment, hosting, serving; Monitoring: workflow management, model output monitoring, model maintenance.  ML Algorithm Types: Deep Learning & Probabilistic Deep learning (DL) Tech giants are already investing heavily in DL algorithms and are pushing innovation forward in the data mining industry. It’s no surprise that, as these systems try to emulate the human brain, they are also becoming known as Neural Networks. After receiving data; sounds, images, or text – multilayered models (just like the neurons in our brains) start to learn from it. Deep Learning systems perform better than traditional ML algorithms when big chunks of data are being used. DL is used for speech recognition, image recognition, and natural language processing (NLP). Buoy Health’s AI-backed symptom-self-checker is a good example of an NLP application becoming instrumental during the ‘new normal’ caused by the Covid19 pandemic. When we became a part of the team, our engineers ensured (not sure if ‘ensured’ is the right word. I need clarification about this) model performance and made it scalable for the long-term. The solution was delivered in a timely fashion. Probabilistic The simplest algorithms are sometimes the most powerful ones. The Naive Bayes classification, one of the most prominent algorithms, works well with a small amount of data. This algorithm describes the probability of an event based on previous conditions related to that event. It does a technical calculation of the probability for each tag-classification and gives the proper one as an output. It is often used for Netflix-like recommendation systems and document classification, by segmenting documents (or emails) into one or more categories. And yes! Spam filters also work using the Naive Bayes algorithm type. Machine Learning Applications [img] Image processing  (facial recognition icon); Text processing (translation icon); Sound processing (subtitles icon); Numerical modeling (probability/estimation icon). Time Series processing (energy use icon); Making Sense of Your Data Text Analysis The text-analysis-algorithm transforms unstructured data into valuable insights. This approach enables leaders to make better decisions through text-classification by; topic, sentiment, or language. That’s how all these chat-bots and virtual-assistants work: they analyze a text to understand the customer and provide some extra value. Text Classification Let’s say a company wants to understand its strong suits by analyzing customer feedback. An algorithm analyzes the content and assigns tags to it. At the start, you pre-define the categories like useful, service, performance, partnership, etc.  And voila! The algorithm displays a set of structured and consistent data! It can be used to classify incoming tech support tickets, spam messages, and positive-negative feedback segmentation. By proper text classification settings, machine learning algorithms can identify emotions and help understand how people feel about an organization. Data Analysis Methods  Anomaly detection – the process of identifying unwanted patterns, such as fraud or intrusion – often used in medical diagnostics, payments screening, weather monitoring, and cyber-security. Clustering – the process of grouping similar patterns to mitigate potential risks – often used to detect fake news, and when an article doesn’t belong to any topic cluster Predictive analytics – ability to make predictions based on previous (historical) data – often used for customer behavior analysis in FinTech (credit scoring calculation), and high traffic zones prediction. (There are many accepted abbreviations for fintech. I chose what I believe to be the clearest to the reader.) Association rule learning – a method of discovering relations between variables in large datasets – often used as a database for recommending system-algorithms and customer-behavior analytics. (Maybe the word method is more appropriate than the word database here?) Machine Learning Benefits for Business [img] Scalability. Train your model once and let it automate all of your routines one-step-at-a-time without hiring additional people. Real-Time Insights. Machine learning models analyze your data in real-time enabling you to tackle risks before they become dangerous. Accuracy. Data-Labeling Algorithms define a set of patterns which they then apply to other never-before-seen datasets, and the results are accurate and seemingly magical. What Does Machine Learning Adoption Look Like? Every Machine Learning application has a lot of moving parts. While proof-of-concept often involves verifying a simple model; in the production stage, several servers are needed to train and monitor different models before integrating them into a solution. This process needs to be fine-tuned to each model since data and challenges will be new each time. (Servers or people?) Proof-of-Concept Production Data-set A single .csv file sampled from production data. Terra-bytes of data from different sources; require standardization before launch Code A few hundred lines of unmaintainable code that unveils the logic of the solution. 20 000+ lines of code running on separate repositories; requires quick access to changes and continuous improvement of processes.  Model Resides on a local machine since it doesn’t require any maintenance. Infrastructure is stored on different databases and orchestrated by tools like Kubernetes for smooth deployment and maintenance. Is ML A Good Idea for My Organization? Machine learning adoption is a good idea if you:  Have a workforce spending a lot of time on routine tasks or activities that can be improved by the use of technology (e.g., support tickets or email marketing) Have to manually gain insights from vast amounts of unstructured data. If that’s the case, make sure your data has been organized and cleansed (or outsource it to an expert) so it can be analyzed by algorithms. See how your competition can be leapfrogged by using advanced data analytics and are ready to start your innovation. Machine learning is likely to be a bad fit if you: Want to have strict ROI measurements Have small amounts of data Don’t have an experienced adviser as your partner How To Build a Machine Learning Team? It’s crucial to make sure engineers and managers have a vast knowledge of the domain, as well as a background in the industry. An efficient team for getting solutions into production might look like this: Basic Team Smart Team Researcher/Scientist + Machine Learning Engineer Domain Expert + Technical Project Manager + Machine Learning Engineer + DevOps Engineer Where Do Most ML Projects Fail? It’s often hard to win customers over if they can’t be shown how a model works (even if this model is brilliant). That’s why before a machine learning solution is committed to, understanding all the requirements of a model is vital. That the regulatory side of the project won’t be affected must then be ensured. A consistent pipeline process is also needed to make changes to the model. Yes, it requires constant improvements. Consider partnering with a vendor that has already gone through all the implementation mistakes and knows how to avoid them. You may have a well-working machine-learning model that serves data-sensitive industries. On the other hand, it may drain your budget and doesn’t seem like a cost-efficient solution in the long-term. The reason for this is that some data science vendors build over-complicated models. This approach always leads to the growth of project scope. You can prevent this by partnering with an expert advisor to set a strict, yet simple benchmark for project success. Another dangerous situation is a legacy model. If you trained your algorithm on temporal data from January 2020 it will be inaccurate for January 2021, especially with some “black swans” like a pandemic which affects the healthcare industry. Update and maintain your model at least quarterly to make sure you’re staying competitive with your solution. The next thing to consider is the human factor. There’s a category of people always reluctant to make any changes, and that’s OK. The term ‘Machine Learning’ is over-hyped, as we’ve explored earlier, and people sometimes see it as a threat to their jobs, understandably. Especially those who work with manual data analysis and patterns investigation. Be honest, talk to the team about possible consequences, and handle their objections by focusing on things which machines can’t perform. How to Find the Right Partner for Machine Learning Solution Development? As stated earlier, there’s no silver bullet and every cooperation has many moving parts. In any case, you can choose a partner with the right features by looking for the following virtues: (Is cooperation the best term here? Maybe ‘project’ is better? Also earlier it was stated that ‘ML projects’ have many moving parts – nothing about groups of people.) They dig and probe until the very end. A trusted partner always asks the right questions so that you leave a meeting without any concerns about whether or not they are engaged. Consider whether they learn as much as possible about your project/situation or not. Small miscommunications often lead to huge mistakes later on. They’re not a “yes-man”. Those who simply accept every idea you have are often those who won’t tell you about the risks or absurdities of a solution. We’ve mentioned that not every industry and every project requires an ML solution. Seek out a partner who commits to your success, not to your budget. They showcase their expertise. Although many companies hide their incompetency behind NDAs, a trusted partner is one who lights the way for you, without bravado. For example, we at Newfire Global Partners value privacy as a top priority, especially when it comes to data analytics services. At the same time, we can tell you about companies we have partnered with, and what sort of value we delivered. (I’m not clear about what the 2 parts of this first sentence have to do with each other. NDAs and lighting the way?) To sum things up Machine learning helps many organizations to simplify their routine and to automate their internal processes. Not every ML or AI idea is worth an investment, but having a clear vision of creating value from your data may become your key business driver. A good partner for machine learning solution development will ask you a ton of questions, dive into your domain, and won't be afraid to say “no”. We at Newfire Global Partners help organizations focus on innovation by becoming a part of their teams. If that’s what you’re looking for, let’s see if we’re a good fit. (maybe integrating into their teams is better?)