You are reading the article Follow These Strategies To Win A Machine Learning Hackathon updated in November 2023 on the website Hatcungthantuong.com. We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested December 2023 Follow These Strategies To Win A Machine Learning Hackathon
At the end of every machine learning hackathon, participants will have amazing learningThe tech sphere brings exciting things for tech-savvy people literally every day. Whether you are a beginner or an expert, you can still participate in many tech-related competitions, conferences, events, and seminars to shape your skills. One such effective event to test your talent is the
Strategies to Win a Machine Learning Hackathon Start from Short HackathonsIf you are new to machine learning and want to give it a try in a hackathon, then short or online hackathons should be your first choice. Remember victory comes with experience. Therefore, directly jumping into long hackathons won’t secure you the winner title. Start from small which are below 24-hours and then move on to long hackathons. But make sure you are well-organized and prepared when you shift from short to long competitions.
Join Hands with an Experienced PersonAs a beginner, you should try to enter a hackathon with someone who is experienced and has great knowledge about machine learning. This will help you both learn throughout the process and win the prize if possible. Besides, make sure you join hands with somebody who can contradict your basic knowledge. For example, if you are a developer, join a league with a person who has business knowledge. With the combination of business and developing skills, you can surely bank the first place.
Formulate a Diverse TeamDiverse here doesn’t mean ideology but talent. If all the members of the team are developers and have zero knowledge about other perspectives, then the result will completely be a slider on one side. Therefore, you should ensure that everybody has something new and different from others to offer in your team. Also, try to go with clients who can support you throughout the challenge. Ask them to constantly check your development and give directions if necessary.
Sort Out Your GoalSuccess in a hackathon comes in two different ways. One is winning the competition and the other is impressing the client. Therefore, before beginning the work, sort out which one you are going to concentrate on. If you are planning to impress the judges and win the hackathon, then you should go with shiny software and an outstanding presentation. But it is quite opposite to impress the client. You should come up with useful software that can be utilized even after the competition is over if you planning to impress your sponsor.
Prepare the Perfect DatasetThe tech sphere brings exciting things for tech-savvy people literally every day. Whether you are a beginner or an expert, you can still participate in many tech-related competitions, conferences, events, and seminars to shape your skills. One such effective event to test your talent is the machine learning hackathon . Today, machine learning is emerging as a powerful technology behind every digital mechanism. Many people are aspiring to become well-versed in machine learning . However, they can enhance their practical skills by participating in machine learning hackathons. Machine learning hackathons are conducted specifically for programmers , coders, and others involved in the development of software. Other professionals like interface designers, project managers, domain experts, graphic designers, and others who work closely on software development will also try their hand at the competition. During a hackathon, participants will be asked to create a working software with the datasets and models provided within a limited time. Fortunately, at the end of every machine learning hackathon , participants will have amazing learning. However, winning a hackathon is completely different from participating in one for the experience. If you are planning to be the star of the event, then you should follow certain strategies to win a machine learning hackathon If you are new to machine learning and want to give it a try in a hackathon, then short or online hackathons should be your first choice. Remember victory comes with experience. Therefore, directly jumping into long hackathons won’t secure you the winner title. Start from small which are below 24-hours and then move on to long hackathons. But make sure you are well-organized and prepared when you shift from short to long chúng tôi a beginner, you should try to enter a hackathon with someone who is experienced and has great knowledge about machine learning. This will help you both learn throughout the process and win the prize if possible. Besides, make sure you join hands with somebody who can contradict your basic knowledge. For example, if you are a developer, join a league with a person who has business knowledge. With the combination of business and developing skills, you can surely bank the first place.Diverse here doesn’t mean ideology but talent. If all the members of the team are developers and have zero knowledge about other perspectives, then the result will completely be a slider on one side. Therefore, you should ensure that everybody has something new and different from others to offer in your team. Also, try to go with clients who can support you throughout the challenge. Ask them to constantly check your development and give directions if necessary.Success in a hackathon comes in two different ways. One is winning the competition and the other is impressing the client. Therefore, before beginning the work, sort out which one you are going to concentrate on. If you are planning to impress the judges and win the hackathon, then you should go with shiny software and an outstanding presentation. But it is quite opposite to impress the client. You should come up with useful software that can be utilized even after the competition is over if you planning to impress your chúng tôi is the core of software development along with the coding. Therefore, always make sure you are preparing the data well before starting core operations. But scraping data is time-consuming and can be tricky when it comes to dynamically generated content. Instead, try to go with publicly available data provided by IMDb or maybe the Kaggle dataset. One thing to keep in mind is that many winning teams usually save their time by seeking readily available data.
You're reading Follow These Strategies To Win A Machine Learning Hackathon
A Super Quick Introduction To Machine Learning
Introduction.
As we enter future times, man is trying his best to uncover all the secrets of technology. Looking back at a famous quote that was once said by Mark Cuban:
“Artificial Intelligence, deep learning, machine learning- whatever you’re doing if you don’t understand it- learn it. Because otherwise, you’re going to be a dinosaur within 3 years“.
The aforementioned words tell us just how rapidly we have evolved in technology. However, in modern times, the race to create world-changing technologies has never been so real. Every being in this world wants to create a technology that is said to out-do a human a hundred-fold- it is said to become the beginning of a new era in which a machine could possess the ability to govern the world, the era in which world peace could be threatened. It is known as ASI- an Artificial Super Intelligence. It’s the basis- Machine Learning. Let us touch the surface and dive into the depths of Machine Learning.
Source: Forbes
The history of Machine LearningGoing back a few decades into the past, Machine Learning is thought to have originated around the early 1960s, a few years after the first computer systems were invented. The term “Machine” represents a computer, mobile phone, or any device that comprises electric circuits and which needs electrical power or a power supply to function. “Learning” refers to an object’s ability to isolate, discover and replicate patterns and use these to carry out tasks- much like how the human being functions.
Machine Learning is often abbreviated as ML. The computational power and benefits of ML were not brought to light until recent times. As time goes on, the versatility of ML is only expanding.
ML in the modern days.ML is one of the most trending topics in the fourth industrial revolution. In the field of Machine Learning, we make use of programmers(developers) and programming languages to conduct the teaching and learning processes, respectively. Examples of popular Machine Learning programming languages include, but are not limited to, Python, Java, C++, C#, JavaScript. The main aim of ML is to teach the computer how to use its past experiences to perform better in the future.
It is essential to know that in Machine Learning, a programmer will not explicitly hard code all possible program outcomes. Instead, the developer will program the application’s ability to learn- Based on what this application has learned previously, it utilizes this to decide on outcomes for present or future problems. The point of reference that a machine uses for learning and is used to learn from is known as a model.
Source: Depositphotos
In ML, the model is generated by algorithms. An algorithm is simply a set of rules/instructions that is followed by a computer or any object when it is performing a certain task. These algorithms have parameters, which are basically the “settings”. By us tampering with the “settings”, we can adjust how the computer learns from the data at hand, and ultimately, we are changing the model that is being compiled as the outcome.
So, we now know that changing the values of the parameters, will cause a different model to be generated.
The Machine Learning frameworkBefore we discuss the different types of Machine Learning, we must understand the learning framework that exists around Machine Learning, while using an example scenario. Suppose you visit a store that sells varieties of grapes and it is the first time you ever set foot into the store.
Since you have no experience with the nature of grapes that the store sells, you decide to taste all the grapes and label them in groups according to their physical properties such as color. After a repeated tasting of the goods, you reach conclusions of your own as to which grape is suitable for you to buy. You have concluded your tasting and are now know which grapes will be sweet or sour.
Looking back at our example, we see that the conclusion you have reached is based on a sequence of events that occurred during the duration of the training and testing stages. Keeping the latter statement in mind, the Machine Learning framework is comprised of ML Input, Data Model, ML Output, and Measure of Success.
Identifying our framework events using the previous scenario, our Machine Learning event will be as follows:
1. Machine Learning Input: This is the first part of the ML framework that we encounter. For this, we had to taste all the grapes, find a fixed set of grape properties, and focus on them for the duration of the problem, for it will be these properties that will assist us in choosing better grapes each time we sample an individual grape. During the tasting of the grapes, we decided to label them into groups based on their physical properties and taste. The grapes that taste good are the ones that we were aiming to reach and therefore this group will be called the “training’ group. It is because each time we sample a grape, the ‘training’ group of grapes will help us to make better predictions as to whether a grape will be sweet or sour or not.
2. Data Model: Upon tasting sufficient grapes, we have grown the groups of grapes large enough (e.g., Group one has 10 grapes all of which are red-yellow and taste sweet; Group 2 has 10 grapes all of which are green-blue and taste sour). With enough data in our arsenal for sweet and sour grapes, we can confirm that we are gradually becoming increasingly aware of which grapes are sweet and which are sour, therefore reinforcing the fact that we are gaining experience and are building up a model which we are using as a point of reference for learning. It is this model that helps us to tell beforehand if a grape will be sweet or sour. With a model, you will have assistance with selecting the sweet and sour grapes- even if it is your first time.
3. Machine Learning Output: With the model, you have created, you would like to have sound output. The output part of the Machine Learning framework is what involves authenticating that your model is true. During the initial stages of the process, you did not possess any experience to assist you with the classification of sweet and sour grapes. However, after training, you have gained enough experience to make you aware of predicting which grape may be sweet and which may be sour. This part of the framework enables you to, with some certainty, decide beforehand whether a grape will be sweet or sour.
4. Measure of Success: This is referring to the level of accuracy that you have achieved. If you were to allow your model to guide you entirely while conducting a grape taste, is your model’s guidance correct? Does the grape taste exactly how the model says it should taste? It is important to note that the success of the model in our scenario, does not necessarily mean you choose sweet grapes all the time. The success factor is dependent on and defined by what you want to see as the outcome at the end. If you are looking for sweet grapes then that is your measure of success, however, if you are looking for sour grapes then that will be it.
Source: NDTV Food
Different types of Machine Learning.is used to teach the model how to make predictions is called the Features Matrix (Matrix because it has more than one dimension). The portion of the information that we want to predict is called the Target Vector (Vector because it is one-dimensional).
You may often see that the Target Vector is referred to as ‘Labels’. Our algorithm will make use of the Features Matrix and try to figure out the best possible way to teach the model to achieve the most accurate prediction results. Many different types of Machine Learning exist, with four main categories. Each type of Machine Learning has been named according to the nature and type of algorithms that fall within the specificcategory.
Category 1- Supervised Machine Learning.In this type of ML, we have data that is passed to the learning algorithms and it is composed of a features matrix and a target vector. Supervised Machine Learning consists of the following algorithms: Linear Regression, K-Nearest Neighbors, Naïve Bayes, Decision Trees, Random Forests, Logistic Regression, Support Vector Machines (SVMs).
Category 2- Unsupervised Machine Learning.This category of Machine Learning consists of Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Clustering algorithm, Markov algorithm, and Neural Networks. When we deal with these algorithms, we pass in a features matrix only. Hence, this means that we do not pass in a target vector. Without a target vector, the unsupervised machine learning algorithms do not have any experience with how to use the features matrix to reach the result, and so these algorithms are left to draw their own conclusions from the data.
For understanding purposes, we shall have a brief explanation of Neural Networks. Specifically, we will talk about Feedforward Neural Networks- so named because data is ‘fed forward’, deeper and deeper into the network-, also known as Multilayer Perceptrons or MLPs. Within this Neural Network there exist three types of layers. At the start of the network, there is an Input Layer. At the end of the network, there is an Output Layer. Between the Input- and Output-Layer, there is a Hidden Layer. The interesting part is that there is no fixture on the number of hidden layers that need to exist in the Neural Network. So, what we are saying is that there may exist an unlimited number of hidden layers between the input and output layer. These neural networks with many hidden layers (e.g., 100), are referred to as ‘deep’ neural networks, and their field is known as ‘Deep Learning’.
Category 3- Semi-Supervised Machine Learning. Category 4- Reinforcement Learning.This category of Machine Learning is based on an “action reward” system. The main aim of reinforcement learning is to teach the machine correct from incorrect. There are three main principles in reinforcement learning.
1. Present State- this describes the current position of the machine.
2. Action- this describes the action that the machine needs to take.
3. Reward- this is the reward that will be given to the machine if it has made the correct decision or provided the expected outcome. For example, giving a binary one (1) will indicate to the machine that it is correct, hence enabling the machine to perform better the next time; and giving a zero (0) will indicate incorrect performance.
When a machine acts in its present state, it will receive a reward if it has succeeded in its attempt.
Source: CFA institute
This concludes my article on Machine Learning.
Thank you for your time.
Related
Introduction To Online Machine Learning: Simplified
Data is being generated in huge quantities everywhere. Twitter generates 12 + TB of data every day, Facebook generates 25 + TB of data everyday and Google generates much more than these quantities everyday. Given that such data is being produced everyday, we need to build tools to handle data with high
1. Volume : High volume of data are stored today for any industry. Conventional models on such huge data are infeasible.
2. Velocity : Data come at high speed and demand quicker learning algorithms.
3. Variety : Different sources of data have different structures. All these data contribute to prediction. A good algorithm can take in such variety of data.
A simple predictive algorithm like Random Forest on about 50 thousand data points and 100 dimensions take 10 minutes to execute on a 12 GB RAM machine. Problems with hundreds of millions of observation is simply impossible to solve using such machines. Hence, we are left with only two options : Use a stronger machine or change the way a predictive algorithm works. First option is not always feasible. In this article we will learn about On-line Learning algorithms which are meant to handle data with such high Volume and Velocity with limited performance machines.
How does On-line learning differ from batch learning algorithms?If you are a starter in the analytics industry, all you would have probably heard of will fall under batch learning category. Let’s try to visualize how the working of the two differ from each other.
Batch learning algorithms take batches of training data to train a model. Then predicts the test sample using the found relationship. Whereas, On-line learning algorithms take an initial guess model and then picks up one-one observation from the training population and recalibrates the weights on each input parameter. Here are a few trade-offs in using the two algorithms.
Computationally much faster and more space efficient. In the online model, you are allowed to make exactly one pass on your data, so these algorithms are typically much faster than their batch learning equivalents, since most batch learning algorithms are multi-pass. Also, since you can’t reconsider your previous examples, you typically do not store them for access later in the learning procedure, meaning that you tend to use a smaller memory footprint.
Usually easier to implement. Since the online model makes one pass over the data, we end up processing one example at a time, sequentially, as they come in from the stream. This usually simplifies the algorithm, if you’re doing so from scratch.
More difficult to maintain in production. Deploying online algorithms in production typically requires that you have something constantly passing datapoints to your algorithm. If your data changes and your feature selectors are no longer producing useful output, or if there is major network latency between the servers of your feature selectors, or one of those servers goes down, or really, any number of other things, your learner tanks and your output is garbage. Making sure all of this is running ok can be a trial.
More difficult to evaluate online. In online learning, we can’t hold out a “test” set for evaluation because we’re making no distributional assumptions — if we picked a set to evaluate, we would be assuming that the test set is representative of the data we’re operating on, and that is a distributional assumption. Since, in the most general case, there’s no way to get a representative set that characterizes your data, your only option (again, in the most general case) is to simply look at how well the algorithm has been doing recently.
Usually more difficult to get “right”. As we saw in the last point, online evaluation of the learner is hard. For similar reasons, it can be very hard to get the algorithm to behave “correctly” on an automatic basis. It can be hard to diagnose whether your algorithm or your infrastructure is misbehaving.
In cases where we deal with huge data, we are left with no choice but to use online learning algorithms. The only other option is to do a batch learning on a smaller sample.
Example Case to understand the conceptWe want to predict the probability that it will rain today. We have a panel of 11 people who predict the class : Rain and non-rain on different parameters. We need to design an algorithm to predict the probability. Let us first initialize a few denotions.
i are individual predictors
w(i) is the weight given to the i th predictor
Initial w(i) for i in [1,11] are all 1
We will predict that it will rain today if,
Once, we have the actual response of the target variable, we now send a feedback on the weights of all the parameters. In this case we will take a very simple feedback mechanism. For every right prediction, we will keep the weight of the predictor same. While for every wrong prediction, we divide the weight of the predictor by 1.2 (learning rate). With time we expect the model to converge with a right set of chúng tôi created a simulation with 1000 predictions done by each of the 11 predictors. Here is how our accuracy curve came out,
Each observation was taken at a time to re adjust the weights. Same way we will make predictions for the future data points.
End NotesOnline learning algorithms are widely used by E-commerce and social networking industry. It is not only fast but also has the capability to capture any new trend visible in with time. A variety of feedback systems and converging algorithms are presently available which should be selected as per the requirements. In some of the following articles, we will also take up a few practical examples of Online learning algorithm applications.
Did you find the article useful? Have you used online learning algorithms before ? Share with us any such experiences. Do let us know your thoughts about this article in the box below.
If you like what you just read & want to continue your analytics learning, subscribe to our emails, follow us on twitter or like our facebook page.Related
Why Is Machine Learning Important?
Machine learning can be considered a component of artificial intelligence and involves training the machine to be more intelligent in its operations. AI technology focuses on incorporating human intelligence while machine learning is focused on making the machines learn faster. So we can say that machine learning engineers can provide faster and better optimizations to AI solutions.
AI technology has had a massive impact on society and has transformed almost every industrial sector from planning to production. Thus machine learning engineers and experts are also of great value to this growing industry.
Why is Machine Learning So Useful?Machine learning is comparatively new but it has existed for many years. Recently gaining a lot of attention, it is essential for many significant technological improvements.
When it comes to business operations, you can access a lot of data with the help of machine learning algorithms. Machine learning also offers more affordable data storage options that have made big data sets possible and accessible for organizations. It has also helped maximize the processing power of computers to be able to perform calculations and operations faster.
Wherever you find AI technology, you will find machine learning experts working to improve the efficiency and results of the AI technologies and machines involved.
Where can Machine Learning be Applied?Machine learning has a lot of applications in a variety of tasks and operations. It plays a central role in the collection, analysis, and processing the large sets of data. It is not just restricted to the businesses and organizations, you have already interacted with them. However, you might not be aware of the fact that you have already been using machine learning technology. Here are a few examples you can relate to as part of our daily lives.
Machine learning solutions are being incorporated into the medical sciences for better detection and diagnosis of diseases. Here is the interesting part. Machine learning can even be used to keep a check on the emotional states with the help of a smartphone.
This technology is also widely used by manufacturers to minimize losses during operations and maximize production while reducing the cost of maintenances through timely predictions.
The banking industry is also utilizing machine learning to identify any fraudulent practices or transactions to avoid losses. Machine learning can also be used to give significant insights into financial data. This in turn results in better investments and better trades.
When it comes to transportation, the self-driving cars of Google or Tesla are powered by Lachine learning. Thus it can be extremely beneficial for autonomous driving and better interpretations.
What do Machine Learning Engineers do?Why Pursue a Career in Machine Learning?
There are many reasons to pursue a career in machine learning. It is not only getting [popular and high in demand, but also an interesting discipline where you can be innovative once you have acquired the necessary skills.
Wrapping Up
The aforementioned discussion describes the significant role of the growing machine learning and AI technology in the industrial and business sector and why you should consider pursuing a career in it.
10 Automated Machine Learning For Supervised Learning (Part 2)
This article was published as a part of the Data Science Blogathon
IntroductionThis post will discuss 10 Automated Machine Learning (autoML) packages that we can run in Python. If you are tired of running lots of Machine Learning algorithms just to find the best one, this post might be what you are looking for. This post is the second part of this first post. The first part explains the general concept of Machine Learning from defining the objective, pre-processing, model creation and selection, hyperparameter-tuning, and model evaluation. At the end of that post, Auto-Sklearn is introduced as an autoML. If you are already familiar with Machine Learning, you can skip that part 1.
The main point of the first part is that we require a relatively long time and many lines of code to run all of the classification or regression algorithms before finally selecting or ensembling the best models. Instead, we can run AutoML to automatically search for the best model with a much shorter time and, of course, less code. Please find my notebooks for conventional Machine Learning algorithms for regression (predicting house prices) and classification (predicting poverty level classes) tasks in the table below. My AutoML notebooks are also in the table below. Note that this post will focus only on regression and classification AutoML while AutoML also can be applied for image, NLP or text, and time series forecasting.
Now, we will start discussing the 10 AutoML packages that can replace those long notebooks.
AutoSklearnThis autoML, as mentioned above, has been discussed before. Let’s do a quick recap. Below is the code for searching for the best model in 3 minutes. More details are explained in the first part.
Regression
!apt install -y build-essential swig curl !pip install auto-sklearn !pip install scipy==1.7.0 !pip install -U scikit-learn from autosklearn.regression import AutoSklearnRegressor from sklearn.metrics import mean_squared_error, mean_absolute_errorRunning AutoSklearn
# Create the model sklearn = AutoSklearnRegressor(time_left_for_this_task=3*60, per_run_time_limit=30, n_jobs=-1) # Fit the training data sklearn.fit(X_train, y_train) # Sprint Statistics print(sklearn.sprint_statistics()) # Predict the validation data pred_sklearn = sklearn.predict(X_val) # Compute the RMSE rmse_sklearn=MSE(y_val, pred_sklearn)**0.5 print('RMSE: ' + str(rmse_sklearn))Output:
auto-sklearn results: Dataset name: c40b5794-fa4a-11eb-8116-0242ac130202 Metric: r2 Best validation score: 0.888788 Number of target algorithm runs: 37 Number of successful target algorithm runs: 23 Number of crashed target algorithm runs: 0 Number of target algorithms that exceeded the time limit: 8 Number of target algorithms that exceeded the memory limit: 6 RMSE: 27437.715258009852Result:
# Scatter plot true and predicted values plt.scatter(pred_sklearn, y_val, alpha=0.2) plt.xlabel('predicted') plt.ylabel('true value') plt.text(100000, 400000, 'RMSE: ' + str(round(rmse_sklearn))) plt.text(100000, 350000, 'MAE: ' + str(round(mean_absolute_error(y_val, pred_sklearn)))) plt.text(100000, 300000, 'R: ' + str(round(np.corrcoef(y_val, pred_sklearn)[0,1],4))) plt.show()Fig. 1 AutoSklearn Regression Result
Classification
Applying AutoML
!apt install -y build-essential swig curl from autosklearn.classification import AutoSklearnClassifier # Create the model sklearn = AutoSklearnClassifier(time_left_for_this_task=3*60, per_run_time_limit=15, n_jobs=-1) # Fit the training data sklearn.fit(X_train, y_train) # Sprint Statistics print(sklearn.sprint_statistics()) # Predict the validation data pred_sklearn = sklearn.predict(X_val) # Compute the accuracy print('Accuracy: ' + str(accuracy_score(y_val, pred_sklearn)))Output:
auto-sklearn results: Dataset name: 576d4f50-c85b-11eb-802c-0242ac130202 Metric: accuracy Best validation score: 0.917922 Number of target algorithm runs: 40 Number of successful target algorithm runs: 8 Number of crashed target algorithm runs: 0 Number of target algorithms that exceeded the time limit: 28 Number of target algorithms that exceeded the memory limit: 4 Accuracy: 0.923600209314495Result:
# Prediction results print('Confusion Matrix') print(pd.DataFrame(confusion_matrix(y_val, pred_sklearn), index=[1,2,3,4], columns=[1,2,3,4])) print('') print('Classification Report') print(classification_report(y_val, pred_sklearn))Output:
Confusion Matrix 1 2 3 4 1 123 14 3 11 2 17 273 11 18 3 3 17 195 27 4 5 15 5 1174 Classification Report precision recall f1-score support 1 0.83 0.81 0.82 151 2 0.86 0.86 0.86 319 3 0.91 0.81 0.86 242 4 0.95 0.98 0.97 1199 accuracy 0.92 1911 macro avg 0.89 0.86 0.88 1911 weighted avg 0.92 0.92 0.92 1911 Tree-based Pipeline Optimization Tool (TPOT)TPOT is built on top of scikit-learn. TPOT uses a genetic algorithm to search for the best model according to the “generations” and “population size”. The higher the two parameters are set, the longer will it take time. Unlike AutoSklearn, we do not set the specific running time for TPOT. As its name suggests, after the TPOT is run, it exports lines of code containing a pipeline from importing packages, splitting the dataset, creating the tuned model, fitting the model, and finally predicting the validation dataset. The pipeline is exported in .py format.
In the code below, I set the generation and population_size to be 5. The output gives 5 generations with increasing “scoring”. I set the scoring to be “neg_mean_absolute_error” and” accuracy” for regression and classification tasks respectively. Neg_mean_absolute_error means Mean Absolute Error (MAE) in negative form. The algorithm chooses the highest scoring value. Making the MAE negative will make the algorithm selecting the MAE closest to zero.
Regression
from tpot import TPOTRegressor # Create model cv = RepeatedStratifiedKFold(n_splits=3, n_repeats=3, random_state=123) tpot = TPOTRegressor(generations=5, population_size=5, cv=cv, scoring='neg_mean_absolute_error', verbosity=2, random_state=123, n_jobs=-1) # Fit the training data tpot.fit(X_train, y_train) # Export the result tpot.export('tpot_model.py')Output:
Generation 1 - Current best internal CV score: -20390.588131563232 Generation 2 - Current best internal CV score: -19654.82630417806 Generation 3 - Current best internal CV score: -19312.09139004322 Generation 4 - Current best internal CV score: -19312.09139004322 Generation 5 - Current best internal CV score: -18752.921100941825 Best pipeline: RandomForestRegressor(input_matrix, bootstrap=True, max_features=0.25, min_samples_leaf=3, min_samples_split=2, n_estimators=100)Classification
from tpot import TPOTClassifier # TPOT that are stopped earlier. It still gives temporary best pipeline. # Create the model cv = RepeatedStratifiedKFold(n_splits=3, n_repeats=3, random_state=123) tpot = TPOTClassifier(generations=5, population_size=5, cv=cv, scoring='accuracy', verbosity=2, random_state=123, n_jobs=-1) # Fit the training data tpot.fit(X_train, y_train) # Export the result tpot.export('tpot_model.py')Output:
Generation 1 - Current best internal CV score: 0.7432273262661955 Generation 2 - Current best internal CV score: 0.843824979278454 Generation 3 - Current best internal CV score: 0.8545565589146273 Generation 4 - Current best internal CV score: 0.8545565589146273 Generation 5 - Current best internal CV score: 0.859616978580465 Best pipeline: RandomForestClassifier(GradientBoostingClassifier(input_matrix, learning_rate=0.001, max_depth=2, max_features=0.7000000000000001, min_samples_leaf=1, min_samples_split=19, n_estimators=100, subsample=0.15000000000000002), bootstrap=True, criterion=gini, max_features=0.8500000000000001, min_samples_leaf=4, min_samples_split=12, n_estimators=100)AutoSklearn gives the result of RandomForestRegressor for the regression task. As for the classification, it gives the stacking of GradientBoostingClassifier and RandomForestClassifier. All algorithms already have their hyperparameters tuned.
Here is to see the validation data scoring metrics.
Regression
pred_tpot = results # Scatter plot true and predicted values plt.scatter(pred_tpot, y_val, alpha=0.2) plt.xlabel('predicted') plt.ylabel('true value') plt.text(100000, 400000, 'RMSE: ' + str(round(MSE(y_val, pred_tpot)**0.5))) plt.text(100000, 350000, 'MAE: ' + str(round(mean_absolute_error(y_val, pred_tpot)))) plt.text(100000, 300000, 'R: ' + str(round(np.corrcoef(y_val, pred_tpot)[0,1],4))) plt.show()Output:
Fig. 2 TPOT regression result.
Classification
pred_tpot = results # Compute the accuracy print('Accuracy: ' + str(accuracy_score(y_val, pred_tpot))) print('') # Prediction results print('Confusion Matrix') print(pd.DataFrame(confusion_matrix(y_val, pred_tpot), index=[1,2,3,4], columns=[1,2,3,4])) print('') print('Classification Report') print(classification_report(y_val, pred_tpot))Output:
Accuracy: 0.9246467817896389 Confusion Matrix 1 2 3 4 1 117 11 7 16 2 6 288 10 15 3 2 18 186 36 4 5 12 6 1176 Classification Report precision recall f1-score support 1 0.90 0.77 0.83 151 2 0.88 0.90 0.89 319 3 0.89 0.77 0.82 242 4 0.95 0.98 0.96 1199 accuracy 0.92 1911 macro avg 0.90 0.86 0.88 1911 weighted avg 0.92 0.92 0.92 1911 Distributed Asynchronous Hyper-parameter Optimization (Hyperopt)Hyperopt is also usually used to optimize the hyperparameters of one model that has been specified. For example, we decide to apply Random Forest and then run hyperopt to find the optimal hyperparameters for the Random Forest. My previous post discussed it. But, this post is different in that it uses hyperopt to search for the best Machine Learning model automatically, not just tuning the hyperparameters. The code is similar but different.
The code below shows how to use hyperopt to run AutoML. Max evaluation of 50 and trial timeout of 20 seconds is set. These will determine how long the AutoML will work. Like in TPOT, we do not set the time limit in hyperopt.
Regression
from hpsklearn import HyperoptEstimator from hpsklearn import any_regressor from hpsklearn import any_preprocessing from hyperopt import tpe from sklearn.metrics import mean_squared_error # Create the model hyperopt = HyperoptEstimator(regressor=any_regressor(‘reg’), preprocessing=any_preprocessing(‘pre’), loss_fn=mean_squared_error, algo=tpe.suggest, max_evals=50, trial_timeout=20) # Fit the data hyperopt.fit(X_train, y_train)
Classification
from hpsklearn import HyperoptEstimator from hpsklearn import any_classifier from hpsklearn import any_preprocessing from hyperopt import tpe # Create the model hyperopt = HyperoptEstimator(classifier=any_classifier(‘cla’), preprocessing=any_preprocessing(‘pre’), algo=tpe.suggest, max_evals=50, trial_timeout=30) # Fit the training data hyperopt.fit(X_train_ar, y_train_ar)
In the Kaggle notebook (in the table above), every time I finished performing the fitting and predicting, I will show the prediction of validation data results in scatter plot, confusion matrix, and classification report. The code is always almost the same with a little adjustment. So, from this point onwards, I am not going to put them in this post. But, the Kaggle notebook provides them.
To see the algorithm from the AutoML search result. Use the below code. The results are ExtraTreeClassfier and XGBRegressor. Observe that it also searches for the preprocessing technics, such as standard scaler and normalizer.
# Show the models print(hyperopt.best_model())Regression
{'learner': XGBRegressor(base_score=0.5, booster='gbtree', colsample_bylevel=0.6209369845565308, colsample_bynode=1, colsample_bytree=0.6350745975782562, gamma=0.07330922089021298, gpu_id=-1, importance_type='gain', interaction_constraints='', learning_rate=0.0040826994703554555, max_delta_step=0, max_depth=10, min_child_weight=1, missing=nan, monotone_constraints='()', n_estimators=2600, n_jobs=1, num_parallel_tree=1, objective='reg:linear', random_state=3, reg_alpha=0.4669165283261672, reg_lambda=2.2280355282357056, scale_pos_weight=1, seed=3, subsample=0.7295609371405459, tree_method='exact', validate_parameters=1, verbosity=None), 'preprocs': (Normalizer(norm='l1'),), 'ex_preprocs': ()}Classification
{'learner': ExtraTreesClassifier(bootstrap=True, max_features='sqrt', n_estimators=308, n_jobs=1, random_state=1, verbose=False), 'preprocs': (StandardScaler(with_std=False),), 'ex_preprocs': ()} AutoKerasAutoKeras, as you might guess, is an autoML specializing in Deep Learning or Neural networks. The “Keras” in the name gives the clue. AutoKeras helps in finding the best neural network architecture and hyperparameters for the prediction model. Unlike the other AutoML, AutoKeras does not consider tree-based, distance-based, or other Machine Learning algorithms.
Deep Learning is challenging not only for the hyperparameter-tuning but also for the architecture setting. Many ask about how many neurons or layers are the best to use. There is no clear answer to that. Conventionally, users must try to run and evaluate their Deep Learning architectures one by one before finally decide which one is the best. It takes a long time and resources to accomplish. I wrote a post describing this here. However, AutoKeras can solve this problem.
To apply the AutoKeras, I set the max_trials to be 8 and it will try to find the best deep learning architecture for a maximum of 8 trials. Set the epoch while fitting the training dataset and it will determine the accuracy of the model.
Regression
!pip install autokeras import autokeras # Create the model keras = autokeras.StructuredDataRegressor(max_trials=8) # Fit the training dataset keras.fit(X_train, y_train, epochs=100) # Predict the validation data pred_keras = keras.predict(X_val)Classification
!pip install autokeras import autokeras # Create the model keras = autokeras.StructuredDataClassifier(max_trials=8) # Fit the training dataset keras.fit(X_train, y_train, epochs=100) # Predict the validation data pred_keras = keras.predict(X_val) # Compute the accuracy print('Accuracy: ' + str(accuracy_score(y_val, pred_keras)))To find the architecture of the AutoKeras search, use the following code.
# Show the built models keras_export = keras.export_model() keras_export.summary()Regression
Model: "model" Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) [(None, 20)] 0 multi_category_encoding (Mul (None, 20) 0 dense (Dense) (None, 512) 10752 re_lu (ReLU) (None, 512) 0 dense_1 (Dense) (None, 32) 16416 re_lu_1 (ReLU) (None, 32) 0 regression_head_1 (Dense) (None, 1) 33 ================================================================= Total params: 27,201 Trainable params: 27,201 Non-trainable params: 0Classification
Model: "model" Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) [(None, 71)] 0 multi_category_encoding (Mul (None, 71) 0 normalization (Normalization (None, 71) 143 dense (Dense) (None, 512) 36864 re_lu (ReLU) (None, 512) 0 dense_1 (Dense) (None, 32) 16416 re_lu_1 (ReLU) (None, 32) 0 dropout (Dropout) (None, 32) 0 dense_2 (Dense) (None, 4) 132 classification_head_1 (Softm (None, 4) 0 ================================================================= Total params: 53,555 Trainable params: 53,412 Non-trainable params: 143 MLJARMLJAR is another great AutoML. And, you will find why soon. To run MLJAR, I assign the arguments of mode, eval_metric, total_time_limit, and feature_selection. This AutoML will understand whether it is a regression or classification task from the eval_metric. The total_time_limit is the duration of how long we allow MLJAR to run in seconds. In this case, it will take 300 seconds or 5 minutes to find the possible best model. We can also specify whether to allow feature selection. The output then will report the used algorithms and how long they take to finish.
Regression
from supervised.automl import AutoML # Create the model mljar = AutoML(mode="Compete", eval_metric="rmse", total_time_limit=300, features_selection=True) # Fit the training data mljar.fit(X_train, y_train) # Predict the training data mljar_pred = mljar.predict(X_val)Classification
from supervised.automl import AutoML # Create the model mljar = AutoML(mode="Compete", eval_metric="accuracy", total_time_limit=300, features_selection=True) # Fit the training data mljar.fit(X_train, y_train) # Predict the training data mljar_pred = mljar.predict(X_val)
The argument “mode” lets us decide what the MLJAR is expected for. There are 4 types of modes to define the purpose of running the MLJAR. In the example code above, the mode of “compete” is used for winning a competition by finding the best model by tuning and ensembling methods. The mode of “optuna” is used to find the best-tuned model with unlimited computation time. The mode of “perform” builds a Machine Learning pipeline for production. The mode of “explain” is used for data explanation.
The result of MLJAR is automatically reported and visualized. Unfortunately, Kaggle does not display the report result after saving. So, below is how it should look. The report compares the MLJAR results for every algorithm. We can see the ensemble methods have the lowest MSE for the regression task and the highest accuracy for the classification task. The increasing number of iteration lowers the MSE for the regression task and improves the accuracy of the classification task. (The leaderboard tables below actually have more rows, but they were cut.)
Regression
Fig. 3 MLJAR Report for regression
Fig. 4 MLJAR for regession
Fig. 5 MLJAR report for classification (1). Fig. 6 MLJAR report for classification (2).
AutoGluonAutoGluon requires users to format the training dataset using TabularDataset to recognize it. Users can then specify the time_limit allocation for AutoGluon to work. In the example code below, I set it to be 120 seconds or 2 minutes.
Regression
!pip install -U pip !pip install -U setuptools wheel !pip install -U "mxnet<2.0.0" !pip install autogluon from autogluon.tabular import TabularDataset, TabularPredictor # Prepare the data Xy_train = X_train.reset_index(drop=True) Xy_train['Target'] = y_train Xy_val = X_val.reset_index(drop=True) Xy_val['Target'] = y_val X_train_gluon = TabularDataset(Xy_train) X_val_gluon = TabularDataset(Xy_val) # Fit the training data gluon = TabularPredictor(label='Target').fit(X_train_gluon, time_limit=120) # Predict the training data gluon_pred = gluon.predict(X_val)Classification
!pip install -U pip !pip install -U setuptools wheel !pip install -U "mxnet<2.0.0" !pip install autogluon from autogluon.tabular import TabularDataset, TabularPredictor # Prepare the data Xy_train = X_train.reset_index(drop=True) Xy_train['Target'] = y_train Xy_val = X_val.reset_index(drop=True) Xy_val['Target'] = y_val X_train_gluon = TabularDataset(Xy_train) X_val_gluon = TabularDataset(Xy_val) # Fit the training data gluon = TabularPredictor(label='Target').fit(X_train_gluon, time_limit=120) # Predict the training data gluon_pred = gluon.predict(X_val)After finishing the task, AutoGluon can report the accuracy of each Machine Learning algorithm it has tried. The report is called leaderboard. The columns below are actually longer, but I cut them for this post.
# Show the models leaderboard = gluon.leaderboard(X_train_gluon) leaderboardRegression
model score_test score_val pred_time_test . . .
0 RandomForestMSE -15385.131260 -23892.159881 0.133275 . . .
1 ExtraTreesMSE -15537.139720 -24981.601931 0.137063 . . .
2 LightGBMLarge -17049.125557 -26269.841824 0.026560 . . .
3 XGBoost -18142.996982 -23573.451829 0.054067 . . .
4 KNeighborsDist -18418.785860 -41132.826848 0.135036 . . .
5 CatBoost -19585.309377 -23910.403833 0.004854 . . .
6 WeightedEnsemble_L2 -20846.144676 -22060.013365 1.169406 . . .
7 LightGBM -23615.121228 -23205.065207 0.024396 . . .
8 LightGBMXT -25261.893395 -24608.580984 0.015091 . . .
9 NeuralNetMXNet -28904.712029 -24104.217749 0.819149 . . .
10 KNeighborsUnif -39243.784302 -39545.869493 0.132839 . . .
11 NeuralNetFastAI -197411.475391 -191261.448480 0.070965 . . .
Classification
model score_test score_val pred_time_test . . .
0 WeightedEnsemble_L2 0.986651 0.963399 3.470253 . . .
1 LightGBM 0.985997 0.958170 0.600316 . . .
2 XGBoost 0.985997 0.956863 0.920570 . . .
3 RandomForestEntr 0.985866 0.954248 0.366476 . . .
4 RandomForestGini 0.985735 0.952941 0.397669 . . .
5 ExtraTreesEntr 0.985735 0.952941 0.398659 . . .
6 ExtraTreesGini 0.985735 0.952941 0.408386 . . .
7 KNeighborsDist 0.985473 0.950327 2.013774 . . .
8 LightGBMXT 0.984034 0.951634 0.683871 . . .
9 NeuralNetFastAI 0.983379 0.947712 0.340936 . . .
10 NeuralNetMXNet 0.982332 0.956863 2.459954 . . .
11 CatBoost 0.976574 0.934641 0.044412 . . .
12 KNeighborsUnif 0.881560 0.769935 1.970972 . . .
13 LightGBMLarge 0.627143 0.627451 0.014708 . . .
H2OSimilar to AutoGluon, H2O requires the training dataset in a certain format, called H2OFrame. To decide how long H2O will work, either max_runtime_secs or max_models must be specified. The names explain what they mean.
Regression
import h2o from h2o.automl import H2OAutoML h2o.init() # Prepare the data Xy_train = X_train.reset_index(drop=True) Xy_train['SalePrice'] = y_train.reset_index(drop=True) Xy_val = X_val.reset_index(drop=True) Xy_val['SalePrice'] = y_val.reset_index(drop=True) # Convert H2O Frame Xy_train_h2o = h2o.H2OFrame(Xy_train) X_val_h2o = h2o.H2OFrame(X_val) # Create the model h2o_model = H2OAutoML(max_runtime_secs=120, seed=123) # Fit the model h2o_model.train(x=Xy_train_h2o.columns, y='SalePrice', training_frame=Xy_train_h2o) # Predict the training data h2o_pred = h2o_model.predict(X_val_h2o)Classification
import h2o from h2o.automl import H2OAutoML h2o.init() # Convert H2O Frame Xy_train_h2o = h2o.H2OFrame(Xy_train) X_val_h2o = h2o.H2OFrame(X_val) Xy_train_h2o['Target'] = Xy_train_h2o['Target'].asfactor() # Create the model h2o_model = H2OAutoML(max_runtime_secs=120, seed=123) # Fit the model h2o_model.train(x=Xy_train_h2o.columns, y='Target', training_frame=Xy_train_h2o) # Predict the training data h2o_pred = h2o_model.predict(X_val_h2o) h2o_predFor the classification task, the prediction result is a multilabel classification result. It gives the probability value for each class. Below is an example of the classification.
predict p1 p2 p3 p4
4 0.0078267 0.0217498 0.0175197 0.952904
4 0.00190617 0.00130162 0.00116375 0.995628
4 0.00548938 0.0156449 0.00867845 0.970187
3 0.00484961 0.0161661 0.970052 0.00893224
2 0.0283297 0.837641 0.0575789 0.0764503
3 0.00141621 0.0022694 0.992301 0.00401299
4 0.00805432 0.0300103 0.0551097 0.906826
H2O reports its result by a simple table showing various scoring metrics of each Machine Learning algorithm.
# Show the model results leaderboard_h2o = h2o.automl.get_leaderboard(h2o_model, extra_columns = 'ALL') leaderboard_h2oRegression output:
model_id mean_residual_deviance rmse mse mae rmsle …
GBM_grid__1_AutoML_20230811_022746_model_17 8.34855e+08 28893.9 8.34855e+08 18395.4 0.154829 …
GBM_1_AutoML_20230811_022746 8.44991e+08 29068.7 8.44991e+08 17954.1 0.149824 …
StackedEnsemble_BestOfFamily_AutoML_20230811_022746 8.53226e+08 29210 8.53226e+08 18046.8 0.149974 …
GBM_grid__1_AutoML_20230811_022746_model_1 8.58066e+08 29292.8 8.58066e+08 17961.7 0.153238 …
GBM_grid__1_AutoML_20230811_022746_model_2 8.91964e+08 29865.8 8.91964e+08 17871.9 0.1504 …
GBM_grid__1_AutoML_20230811_022746_model_10 9.11731e+08 30194.9 9.11731e+08 18342.2 0.153421 …
GBM_grid__1_AutoML_20230811_022746_model_21 9.21185e+08 30351 9.21185e+08 18493.5 0.15413 …
GBM_grid__1_AutoML_20230811_022746_model_8 9.22497e+08 30372.6 9.22497e+08 19124 0.159135 …
GBM_grid__1_AutoML_20230811_022746_model_23 9.22655e+08 30375.2 9.22655e+08 17876.6 0.150722 …
XGBoost_3_AutoML_20230811_022746 9.31315e+08 30517.5 9.31315e+08 19171.1 0.157819 …
Classification
model_id mean_per_class_error logloss rmse mse …
StackedEnsemble_BestOfFamily_AutoML_20230608_143533 0.187252 0.330471 0.309248 0.0956343 …
StackedEnsemble_AllModels_AutoML_20230608_143533 0.187268 0.331742 0.309836 0.0959986 …
DRF_1_AutoML_20230608_143533 0.214386 4.05288 0.376788 0.141969 …
GBM_grid__1_AutoML_20230608_143533_model_1 0.266931 0.528616 0.415268 0.172447 …
XGBoost_grid__1_AutoML_20230608_143533_model_1 0.323726 0.511452 0.409528 0.167713 …
GBM_4_AutoML_20230608_143533 0.368778 1.05257 0.645823 0.417088 …
GBM_grid__1_AutoML_20230608_143533_model_2 0.434227 1.10232 0.663382 0.440075 …
GBM_3_AutoML_20230608_143533 0.461059 1.08184 0.655701 0.429944 …
GBM_2_AutoML_20230608_143533 0.481588 1.08175 0.654895 0.428887 …
XGBoost_1_AutoML_20230608_143533 0.487381 1.05534 0.645005 0.416031 …
PyCaretThis is the longest AutoML code that this post is exploring. PyCaret does not need splitting the features (X_train) and label (y_train). So, the below code will only randomly split the training dataset into another training dataset and a validation dataset. Preprocessing, such as filling missing data or feature selection, is also not required. Then, we set up the PyCaret by assigning the data, target variable or label, numeric imputation method, categorical imputation method, whether to use normalization, whether to remove multicollinearity, etc.
Regression
!pip install pycaret from pycaret.regression import * # Generate random numbers val_index = np.random.choice(range(trainSet.shape[0]), round(trainSet.shape[0]*0.2), replace=False) # Split trainSet trainSet1 = trainSet.drop(val_index) trainSet2 = trainSet.iloc[val_index,:] # Create the model caret = setup(data = trainSet1, target='SalePrice', session_id=111, numeric_imputation='mean', categorical_imputation='constant', normalize = True, combine_rare_levels = True, rare_level_threshold = 0.05, remove_multicollinearity = True, multicollinearity_threshold = 0.95)Classification
!pip install pycaret from pycaret.classification import * # Generate random numbers val_index = np.random.choice(range(trainSet.shape[0]), round(trainSet.shape[0]*0.2), replace=False) # Split trainSet trainSet1 = trainSet.drop(val_index) trainSet2 = trainSet.iloc[val_index,:] # Create the model caret = setup(data = trainSet1, target='Target', session_id=123, numeric_imputation='mean', categorical_imputation='constant', normalize = True, combine_rare_levels = True, rare_level_threshold = 0.05, remove_multicollinearity = True, multicollinearity_threshold = 0.95)After that, we can run the PyCaret by specifying how many cross-validations folds we want. The PyCaret for regression gives several models sorting from the best scoring metrics. The top models are Bayesian Ridge, Huber Regressor, Orthogonal Matching Pursuit, Ridge Regression, and Passive-Aggressive Regressor. The scoring metrics are MAE, MSE, RMSE, R2, RMSLE, and MAPE. PyCaret for classification also gives several models. The top models are the Extra Trees Classifier, Random Forest Classifier, Decision Tree Classifier, Extreme Gradient Boosting, and Light Gradient Boosting Machine. The below tables are limited for their rows and columns. Find the complete tables in the Kaggle notebook.
# Show the models caret_models = compare_models(fold=5)Regression
Model MAE MSE RMSE R2 …
br Bayesian Ridge 15940.2956 566705805.8954 23655.0027 0.9059 …
huber Huber Regressor 15204.0960 588342119.6640 23988.3772 0.9033 …
omp Orthogonal Matching Pursuit 16603.0485 599383228.9339 24383.2437 0.9001 …
ridge Ridge Regression 16743.4660 605693331.2000 24543.6840 0.8984 …
par Passive Aggressive Regressor 15629.1539 630122079.3113 24684.8617 0.8972 …
… … … … … … …
Classification
Model Accuracy AUC Recall Prec. …
et Extra Trees Classifier 0.8944 0.9708 0.7912 0.8972 …
rf Random Forest Classifier 0.8634 0.9599 0.7271 0.8709 …
dt Decision Tree Classifier 0.8436 0.8689 0.7724 0.8448 …
xgboost Extreme Gradient Boosting 0.8417 0.9455 0.7098 0.8368 …
lightgbm Light Gradient Boosting Machine 0.8337 0.9433 0.6929 0.8294 …
… … … … … … …
To create the top 5 models, run the following code.
Regression
# Create the top 5 models br = create_model('br', fold=5) huber = create_model('huber', fold=5) omp = create_model('omp', fold=5) ridge = create_model('ridge', fold=5) par = create_model('par', fold=5)Classification
# Create the top 5 models et = create_model('et', fold=5) rf = create_model('rf', fold=5) dt = create_model('dt', fold=5) xgboost = create_model('xgboost', fold=5) lightgbm = create_model('lightgbm', fold=5)To tune the selected model, run the following code.
Regression
# Tune the models, BR: Regression br_tune = tune_model(br, fold=5) # Show the tuned hyperparameters, for example for BR: Regression plot_model(br_tune, plot='parameter')Classification
# Tune the models, LightGBM: Classification lightgbm_tune = tune_model(lightgbm, fold=5) # Show the tuned hyperparameters, for example for LightGBM: Classification plot_model(lightgbm_tune, plot='parameter')PyCaret lets the users manually perform ensemble methods, like Bagging, Boosting, Stacking, or Blending. The below code performs each ensemble method.
Regression
# Bagging BR br_bagging = ensemble_model(br_tune, fold=5) # Boosting BR br_boost = ensemble_model(br_tune, method='Boosting', fold=5) # Stacking with Huber as the meta-model stack = stack_models(caret_models_5, meta_model=huber, fold=5) # Blending top models caret_blend = blend_models(estimator_list=[br_tune,huber_tune,omp_tune,ridge_tune,par_tune])Classification
# Bagging LightGBM lightgbm_bagging = ensemble_model(lightgbm_tune, fold=5) # Boosting LightGBM lightgbm_boost = ensemble_model(lightgbm_tune, method='Boosting', fold=5) # Stacking with ET as the meta-model stack = stack_models(caret_models_5, meta_model=et, fold=5) # Blending top models caret_blend = blend_models(estimator_list=[lightgbm_tune,rf,dt])Now, let’s choose blending models as the predictive models. The following code uses the blending models to predict the validation datasets.
Regression
# Predict the validation data caret_pred = predict_model(caret_blend, data = trainSet2.drop(columns=['SalePrice']))Classification
# Predict the validation data pred_caret = predict_model(caret_blend, data = trainSet2.drop(columns=['Target'])) AutoViMLI run AutoViML in the notebook by assigning many arguments. Just like PyCaret, AutoViML does not need splitting the features (X_train) and label (y_train). Users only need to split the training dataset (as trainSet1) and the validation dataset (as trainSet2). Users can specify other parameters, like scoring parameter, hyperparameter, feature reduction, boosting, binning, and so on.
Regression
!pip install autoviml !pip install shap from autoviml.Auto_ViML import Auto_ViML # Create the model viml, features, train_v, test_v = Auto_ViML(trainSet1, 'SalePrice', trainSet2.drop(columns=['SalePrice']), scoring_parameter='', hyper_param='RS', feature_reduction=True, Boosting_Flag=True, Binning_Flag=False,Add_Poly=0, Stacking_Flag=False, Imbalanced_Flag=True, verbose=1)Classification
!pip install autoviml !pip install shap from autoviml.Auto_ViML import Auto_ViML # Create the model viml, features, train_v, test_v = Auto_ViML(trainSet1, 'Target', trainSet2.drop(columns=['Target']), scoring_parameter='balanced_accuracy', hyper_param='RS', feature_reduction=True, Boosting_Flag=True, Binning_Flag=False,Add_Poly=0, Stacking_Flag=False, Imbalanced_Flag=True, verbose=1)After fitting the training data, we can examine what has been done from the output. For example, we can see that AutoViML was doing preprocessing by filling the missing data. Similar to MLJAR, AutoViML also gives the visualized report. For the regression task, it visualizes the scatter plot of true and predicted values using the XGBoost model. It also plots the prediction residual error in a histogram. From the two plots, we can observe that the model has relatively good accuracy and the residual error is a normal distribution.
The next report is ensemble method trials. We can find this report for both regression and classification tasks. The last graph displays the feature importance. We can see that the most important feature to predict house prices is exterior material quality, followed by overall material and finish quality, and size of the garage respectively.
Fig. 7 Scatter plot of true and predicted values, and residual error histogram. Fig. 8 Feature importances.
As for the classification task, it does not visualize the scatter plot, but the ROC curve for each of the four prediction classes, micro-average, and macro-average as well as the AUC value. We can see that all of the classes have AUC values above 0.98. The next chart reports the iso-f1 curves as the accuracy metric. It also later gives the classification report.
Fig. 9 ROC curves and iso-f1 curves
Average precision score, micro-averaged over all classes: 0.97 Macro F1 score, averaged over all classes: 0.83 ##################################################### precision recall f1-score support 0 0.84 0.97 0.90 963 1 0.51 0.59 0.54 258 2 0.18 0.11 0.14 191 3 0.00 0.00 0.00 118 accuracy 0.73 1530 macro avg 0.38 0.42 0.40 1530 weighted avg 0.64 0.73 0.68 1530 [[938 19 6 0] [106 151 1 0] [ 53 117 21 0] [ 16 12 90 0]]The following code prints out the result of AutoViML search. We can see that the best model is XGBRegressor and XGBClassifier for each task.
vimlRegression
XGBRegressor(base_score=0.5, booster='gbtree', colsample_bylevel=1, colsample_bynode=1, colsample_bytree=0.7, gamma=1, gpu_id=0, grow_policy='depthwise', importance_type='gain', interaction_constraints='', learning_rate=0.1, max_delta_step=0, max_depth=8, max_leaves=0, min_child_weight=1, missing=nan, monotone_constraints='()', n_estimators=100, n_jobs=-1, nthread=-1, num_parallel_tree=1, objective='reg:squarederror', predictor='cpu_predictor', random_state=1, reg_alpha=0.5, reg_lambda=0.5, scale_pos_weight=1, seed=1, subsample=0.7, tree_method='hist', ...)Classification
CalibratedClassifierCV(base_estimator=OneVsRestClassifier(estimator=XGBClassifier(base_score=None, booster='gbtree', colsample_bylevel=None, colsample_bynode=None, colsample_bytree=None, gamma=None, gpu_id=None, importance_type='gain', interaction_constraints=None, learning_rate=None, max_delta_step=None, max_depth=None, min_child_weight=None, missing=nan, monotone_constraints=None, n_estimators=200, n_jobs=-1, nthread=-1, num_parallel_tree=None, objective='binary:logistic', random_state=99, reg_alpha=None, reg_lambda=None, scale_pos_weight=None, subsample=None, tree_method=None, use_label_encoder=True, validate_parameters=None, verbosity=None), n_jobs=None), cv=5, method='isotonic') LightAutoMLNow, we have reached the 10th AutoML. LightAutoML is expected to be light as its name explains. Here, we set the task to be ‘reg’ for regression, ‘multiclass’ for multiclass classification, or ’binary’ for binary classification. We can also set the metrics and losses in the task. I set the timeout to be 3 minutes to let it find the best model. After simple .fit and .predict, we can already get the result.
Regression
!pip install openpyxl from lightautoml.automl.presets.tabular_presets import TabularAutoML from lightautoml.tasks import Task # Create the model light = TabularAutoML(task=Task(‘reg’,), timeout=60*3, cpu_limit=4) train_data = pd.concat([X_train, y_train], axis=1) # Fit the training data train_light = light.fit_predict(train_data, roles = {‘target’: ‘SalePrice’, ‘drop’:[]}) # Predict the validation data pred_light = light.predict(X_val)
Classification
!pip install openpyxl from lightautoml.automl.presets.tabular_presets import TabularAutoML from lightautoml.tasks import Task train_data = pd.concat([X_train, y_train], axis=1) # Create the model light = TabularAutoML(task=Task(‘multiclass’,), timeout=60*3, cpu_limit=4) # Fit the training data train_light = light.fit_predict(train_data, roles = {‘target’: ‘Target’}) # Predict the validation data pred_light = light.predict(X_val)
The results for the classification task are the predicted class and the probability of each of the classes. In other words, the result fulfills multiclass and multilabel classification expectations.
# Convert the prediction result into dataframe pred_light2 = pred_light.data pred_light2 = pd.DataFrame(pred_light2, columns=['4','2','3','1']) pred_light2 = pred_light2[['1','2','3','4']] pred_light2['Pred'] = pred_light2.idxmax(axis=1) pred_light2['Pred'] = pred_light2['Pred'].astype(int) pred_light2.head()1 2 3 4 Pred
0 0.00 0.01 0.00 0.99 4
1 0.00 0.00 0.00 1.00 4
2 0.00 0.04 0.00 0.96 4
3 0.00 0.01 0.98 0.01 3
4 0.02 0.38 0.34 0.27 2
ConclusionWe have discussed 10 AutoML packages now. Please notice that there are still more AutoML not yet discussed in this post. Not only in Python notebook but AutoML also can be found in cloud computing or software. Which AutoML is your favourite? Or, do you have other AutoML in mind?
Fig. 10 AutoML chúng tôi you still going to run 10 conventional Machine Learning or prefer the 10 AutoML packages?
About Author
References
Fig3- Image by Author
Fig4-Image by Author
Fig5-Image by Author
Fig6-Image by Author
Fig10- sklearn, topot, hyperopt, autokeras, mljar, autogluon, h2o, pycaret, autoviml
Connect with me here.
The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.
Related
Win The Web Marketing Championship With These 5 Focuses
Everybody wants to win. And there is almost no stiffer competition than for those 10 (or so) first page rankings on the search engines, particularly Google. Every business engaged in web marketing wants to be on that first page. And while there are millions of competitions to enter (in the form of keywords) not every competition has the audience to make it worth the effort.
In the Olympics, it doesn’t matter if anyone is watching. A gold medal is a gold medal. But online, a gold medal for obscure keywords has no merit or value. What you want is the championship ring, not some participation trophy!
The success of any web marketing campaign is reliant on many things, but there are five primary aspects you have to consider if you truly want to be successful online. In fact, failure to keep these five considerations front of mind will very likely lead to “failure” in your web marketing efforts, regardless of who is overseeing them for you.
1) The Definition of SuccessIn sports, the rules are clear and the goals are well defined. Get the ball through the hoop, bring runners home, or score a touchdown to put points on the board. If there is a violation, the refs call it and penalties are handed out. Web marketing doesn’t have a rule book and the penalties are inconsistent at best.
The closest thing we have to any kind of rules of the game are “best practices,” which can vary depending on who’s saying them and what studies are performed. And the goals? Those are often fluid, depending on the business and what site visitors want.
Both you and your marketing team need to be on board with what your web marketing goals are and what metrics you’ll be using to measure those goals. It does no good to point out how much engagement you’re getting on social media if that’s not what you’re most interested in. By establishing your definitions and metrics for success, you can make sure that everyone is on board and everything being done is working toward achieving those goals.
2) The Baseline Authority of the WebsiteYou may never know exactly what your site’s “authority level” is, though there are many tools that will give you some kind of indication as to where you stand against a competitor. And while these are not perfect metrics, they help you see the road ahead.
If you’re starting out on the low end of the authority spectrum, you’ve got a lot of building to do. You can optimize your site for keywords and make it search engine-friendly but, without building authority, it just isn’t going to take off.
Keep in mind, authority isn’t something you can just “do”, like keyword optimization. It’s an ongoing process of marketing, link building, social media engagement, and providing a good customer experience.
Ultimately, building up that authority takes time and a lot of work. If your site is in a strong position, less work will be required to put you over the top. However, if you’re starting at the bottom of the pile, you’ve got a lot of work ahead to dig your way out and up.
3) The Aggressiveness of Your Web Marketing CampaignThe baseline authority of your site says a lot about how much work needs to be done to make your web marketing campaign successful. But the aggressiveness of your campaign has everything to do with how quickly you’ll be tackling those tasks ahead.
I always find it interesting that—on a scale of 1-10, with 10 being most aggressive—a business that is operating on a level three aggressiveness expects results as if they were operating at a 10. Unfortunately, it just doesn’t work that way.
Let’s say you have 100 web marketing items that need to be addressed. It may be impossible to do all 100 all at once, but if you’re only able to hit one or two items each month, you can see that you have a much longer road ahead than if you were tackling those tasks at a more aggressive 6 or 10 tasks per month.
The rate at which you are able to focus on any given area will play a significant role in how quickly you see the success you’re looking for. If you spread yourself too thin, you’re doing little bits of many things and never taking the time to be great with any of them. But if you focus too much on one area and not the other, then that other can drag down the success of the one thing you’re actually doing well. Sometimes it’s just hard to get ahead!
Obviously, it would be great to operate at a 10 aggressiveness level, but for most businesses, time and budget makes that impossible. This means you have to accept the level at which you can operate, understanding that success may not come as quickly as you hoped.
4) The Authority and Aggressiveness of Your CompetitionYour competition is always just that, your competition. If they aren’t doing any web marketing, then you have a very good chance of beating them with any web marketing efforts you start doing. However, if your competition is doing anything at all to bolster their own web marketing efforts, you’re up against a moving target.
Straying a bit from the football metaphor, it seems that many people think of web marketing as a race where the other cars on the track are not moving. Fire up your engine, put it in gear, and you’ll be in first place in no time at all. But the other cars are moving. Some will be moving slowly, but some are already moving at high speeds. Not only do you need to get up to their speed, but you have to be moving faster than they are if you want to catch up to them, let alone pass them on the track of success.
Ok, let’s say you are moving faster than your competition—that doesn’t mean you’re going to pass them right away, especially if they had a significant head start. So now you have to consider just how far ahead they are. Not only that, but you also have to compare your rate of speed to theirs before you can determine when you’ll be able to overtake them.
Of course, that expectation changes the moment they see you coming in their rear view mirror and decide to kick it into a higher gear. And also don’t forget about the guys behind you that also push the accelerator once they saw you pass. Not only are you struggling to get ahead, you’re struggling to stay ahead as well.
That was quite a long metaphor, but I think you get the point. It would be nice if web marketing was like painting a house. Throw on a couple of coats and you’re good for a few years. Unfortunately, we have competitors in this race that are also looking to win and they are, essentially, standing in the way of your own success.
5) Your Own ExpectationsI’ve seen it so many times before. A web marketing campaign is, by all measures, successful based on the starting authority level of the site, the level of aggressiveness of their campaign, and how their current competitors stack up against them, but the client feels like they are losing. The data shows strong, consistent growth, but the client doesn’t feel like it’s “working”. How does this happen?
For starters, it’s usually due to a lack of defining what success means at the start. But, on top of that, it’s because the client has expectations that don’t line up with the reality of the situation. Whatever expectations you have for the success of your web marketing campaign, you need to make sure those are adjusted to align with each of the considerations above.
Yes, we all want results (and now!), but we what we want and what is truly possible need to be in cahoots together.
It does no good for you or your web marketing team to have false expectations of when “success” will be achieved. You’re just setting yourselves up for disappointment—and possibly pulling the plug on a campaign that is in fact succeeding based on the realities of the situation. Inevitably, bad expectations lead to even worse decisions that do more harm than good.
Assess Your BattlefieldEveryone wants to have a successful web marketing campaign, but only an honest assessment of the battlefield is truly going to help you get from where you are to where you want to be.
The next time you feel like your web marketing campaign is failing, come back to this post and give yourself an assessment.
Are you and your web marketing team operating on the same goals?
Did you have a clear understanding of where you were starting from?
Are you operating at an aggressiveness level that will achieve the success you want? In the time you want? Or were you just setting expectations on budget alone?
Are you keeping an eye on your competitors? And do you have an understanding of how much they are investing in web marketing?
Are you regularly reassessing your expectations?
If you can’t answer yes to those questions on a regular basis, you will ultimately have a “success problem”. To win the web marketing championship, you have to be prepared mentally for how, when, and where it will be achieved. Without that, you’ll likely lose the game before the season has even started.
Let me know below what considerations you think are critical to web marketing success.
Image Credits
Update the detailed information about Follow These Strategies To Win A Machine Learning Hackathon on the Hatcungthantuong.com website. We hope the article's content will meet your needs, and we will regularly update the information to provide you with the fastest and most accurate information. Have a great day!