You are reading the article Google’s Automated Image Captioning & The Key To Artificial “Vision” updated in November 2023 on the website Hatcungthantuong.com. We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested December 2023 Google’s Automated Image Captioning & The Key To Artificial “Vision”
Eyes Can See, but Intelligence “Perceives”Artificial sight has been with us for more than a century. Anything with a camera can see. It’s a very basic sort of thing. But even a blind man can surpass the camera’s understanding of what it is looking at. Until very recently, computers were not able to easily and accurately name the objects found in pictures without very specific parameters. To truly say that a man-made object has “vision” would mean that it at least has a concrete ability to specify what it is looking at, rather than just simply looking at it without gathering any context. This way, the device could potentially react to its environment based on sight, just like we do. Perception is an absolute necessity. Without it, every sense we have is useless.
Perception Through Automatic Image CaptioningAlthough we generally believe that every picture is worth a thousand words, Inception V3 doesn’t necessarily share that opinion. The automatic image captioning software has very few things to say about what it sees, but it at least has a basic concrete understanding of what is contained within the frame presented to it.
With this rudimentary information we have taken a step towards the ability of software to understand visual stimuli. Giving a robot this kind of power would allow it to react to such stimuli, bringing its intelligence to just under the level of most basic aquatic animals. That may not sound like much, but if you take a look at how robots are doing right now (when tested outside their highly restrictive parameters), you’ll find that this would be quite a leap in intelligence compared to the amoebic way in which they can perceive their own surroundings.
What This Means for AI (And Why It’s Far From Perfect)The fact that we now have software that (with 93 percent accuracy) can caption images means that we have somewhat overcome the obstacle of getting computers to make sense of their environments. Of course, that doesn’t mean we’re anywhere near finished in that department. It’s also worth mentioning that the Inception V3 was trained by humans over time and uses the information it “learned” to decipher other images. To have true understanding of one’s environment, one must be able to achieve a more abstract level of perception. Is the person in the image angry? Are two people fighting? What is the woman on the bench crying about?
The above questions represent the kinds of things we ask ourselves when we encounter other human beings. It’s the kind of abstract inquiry that requires us to extrapolate more information than what an image captioning doohickey can do. Let’s not forget that icing on the cake we like to call an emotional (or “irrational”) reaction to what we see. It’s why we consider flowers beautiful, sewers disgusting, and french fries tasty. It’s something we are still wondering whether we will ever achieve on a machine level without actually hard-coding it. The truth is that this kind of “human” phenomenon is likely impossible without restrictive programming. Of course, that doesn’t mean we won’t stop trying. We are, after all, human.
Miguel Leiva-Gomez
Miguel has been a business growth and technology expert for more than a decade and has written software for even longer. From his little castle in Romania, he presents cold and analytical perspectives to things that affect the tech world.
Subscribe to our newsletter!
Our latest tutorials delivered straight to your inbox
Sign up for all newsletters.
By signing up, you agree to our Privacy Policy and European users agree to the data transfer policy. We will not share your data and you can unsubscribe at any time.
You're reading Google’s Automated Image Captioning & The Key To Artificial “Vision”
Automated Workflow Management: A Comprehensive Guide In 2023
~78% of business leaders think remote working and workplace reconfiguration will boost automation and digital transformation.
Knowledge of workload automation is important to enhance workflow management. However, many business leaders claim not to know about workflow automation:
~68% of businesses do not understand digital transformation.
~75% of organizations do not have the right information about the digital workplace.
We have curated this article to fill that knowledge gap. We cover:
What is a workflow?
What is workflow automation?
What is workflow management?
What are the types of workflow automation?
Which workflow automation type should be used?
What is workload automation (WLA) based workflow automation?
What is business process automation (BPA) based workflow automation?
What is robotic process automation (RPA ) based workflow automation?
What is a workflow?A “workflow” is a series of industrial, administrative, or other processes that take a piece of work from start to finish. A workflow is made possible by putting resources into processes that change materials, give services, or process information in a planned way. A workflow can include multiple steps, people, systems, or machines.
Figure 1: A workflow example.
What is workflow automation?Workflow automation is using software to eliminate or reduce the number of manual tasks and increase work efficiency. Some examples of workflow automation are:
Creation and sending out calendar events and invitations.
Sending messages through platforms like WhatsApp and SMS.
Document creation, like sales contracts, to send customers for signature.
Tasks in data workflows, like data transfer between platforms.
Workload distribution based on volume and time.
What is a workflow management system?A workflow management system (WfMS or WFMS) or workflow application is a set of tools that lets you set up, run, and observe a predetermined set of tasks.
What are the types of workflow automation?Workflow automation can be classified into three categories according to the way it functions:
Which workflow automation type should be used?WLA-based workflow automation can be preferred when back-end workflows are automated, such as:
Server updates.
Data management operations like extract-transform-load (ETL) operations.
Automating tasks for file transfer protocol servers.
BPA-based workflow automation can be used in front-end operations that require flexibility, such as:
Travel requests are processed through digital forms on a single platform.
Creating digital time-off requests forms.
Sorting customer queries to departments by using the software interface.
RPA-based workflow automation can be useful in precisely repeatable front-end operations such as:
Data entering for cashier reports like product ID and product quantity.
Creation of PDF invoices and sending them to a specified email address.
User activity auditing to ensure user compliance.
What is WLA-based workflow automation?Workload automation (WLA) software lets back-end business workflows be scheduled, started, and triggered from one platform.
What are the differences between BPA and RPA?In contrast to BPM and RPA, Workload Automation gives more importance to event-driven triggers, situational dependencies, and real-time processing than time-based processing when planning, starting, and running workflows.
To learn more about WLA vs. RPA, check out this quick read.
Why is it important?There are over 14 case studies of WLA where companies benefit from WLA workflow automation. Workload automation can provide the following benefits:
Discover Creatio – the no-code platform for workflow automation and CRM, offering businesses freedom and flexibility. Trusted by thousands of enterprises in more than 100 countries, millions of workflows are launched on the platform every single day.
Watch their short video clip to get a grasp of their services:
How does it work?
Users can log in to their WLA software to create workflows.
Users can choose the application or service to automate a task on the workflow creation screen.
Then, the user can create an event trigger to initiate another business process with the execution of the application.
In the end, the user can set up an alarm or a message to let him or her know when the business process is done or canceled.
Video 1: An example of the use of the WLA tool:
What are some WLA use cases in workflow automation?This section highlights some use cases of workflow automation:
IT department:
To automate ETL workflows: WLA tools can automate ETL task triggering and execution to reduce human error.
To manage data centers: By using WLA tools, IT teams can create workflows to manage data centers from a single platform.
Human Resources:
To automate payrolls: WLA tools can be used in HR to automate payroll management tasks like the calculation of compensations or initiation of payments.
To automate hiring tasks: By using WLA, HR can schedule online job posts, interviews, and emails.
Finance:
To automate know your customers (KYC) tasks: WLA automation can be used in finance to automate KYC workflow tasks in finance like
business existence checks
Address checks
Financial document recordings
Client onboarding and offboarding: WLA technologies can automate client onboarding and offboarding emails and permissions.
What is business process automation-based workflow automation?Business process automation (BPA) uses software to automate business workflows. Excel’s autofill and macro features are some of the first examples of business process automation used in business processes like sales order processing. 36% of businesses automate workflow, and 26% intend to do so.
What is the difference between BPA & RPA?BPA automates the workflows of business processes to make tasks that used to be done by hand more efficient. BPA focuses more on front-end tasks like automating emails and invoice processing. Or creating more open-ended business tasks like situation-based workflow diagrams to assign tasks to employees. Some of the main differences are:
RPA tools focus on automating specific and exactly repeatable tasks, while BPA can automate situation-dependent workflows.
The use of BPA tools is less predictable, and they are slower than RPA chúng tôi tools are faster and more predictable than BPA tools.
Examine this quick read for more information on RPA vs. BPA.
Why is it important?BPA can assist in business process management (BPM) and improvement. BPM involves discovering, modeling, analyzing, measuring, improving, optimizing, and automating processes. Improving business process management is important because it can have the following benefits:
Improve efficiency: Analysts and leaders can see every step, task, and employee in workflows with BPM. In 55 process improvement case studies we collected, 72% of them improved process efficiency with BPM efforts.
Figure 2: In 72% of companies, the use of BPM increased process efficiency.
Motivate change: Business leaders can use BPM tools to document, monitor, and anticipate changes in workflows.
Motivate compliance and security: Managers and analysts can easily document and visualize workflows with BPM to ensure compliance. For example, 19% of executives consider BPM for audit and compliance purposes.
Learn more about business process management benefits and best practices. Compare BPM, process intelligence, and process mining tools through our data-driven and comprehensive lists.
How does it work?Video 2: An application of business process automation software.
What are some BPA use cases?
Document search: BPA software can aid in finding information on platforms or internal systems that your company uses to store documents.
Video 3: An example of a BPA document search:
Diagramming: Business workflow automation BPA can create workflow diagrams with drag-and-drop features. Workflow diagrams help to explain how a business process works with visual workflows. They can show how a business process works from beginning to end. BPA software can:
Support the Business Process Model and Notation (BPMN) graphically or with other diagramming conventions.
Save time by creating workflow diagrams manually with their pre-defined features.
Figure 3: Workflow diagram.
Email management: Service requests from emails and form submissions can be automated using BPA. BPA software can be programmed to process specific emails and forms. These items could include a customer number or the phrase “support ticket.” When an email arrives, the BPA software can start the tasks that the user has specified, such as sending a “thank you” email with the average response time to the request.
Video 4: An exemplary use of BPA for email management.
What is robotic process automation-based workflow automation?Robotic process automation (RPA) is a common way to make specialized agents, or “bots,” that interact with graphical user interface (GUI) elements to complete repetitive, rules-based workflows. They can, for example, be used to move files to specified folders and read and write databases within a specified time frame.
Figure 4: RPA Services business process architecture
Why is it important?RPA is important because it has the following benefits:
On-time process execution: RPA bots can start a task on a specific day and time each month.
Boosts remote work and reduces costs: RPA lets employees orchestrate and monitor automated workflows from a dashboard remotely.
RPA can manage remote desktop automation: RPA can automate remote desktops using their user interface.
To learn more about the benefit of RPA, check out this quick read.
How does it work?RPA software can require programmers to create and automate bots. On the RPA software:
Video 5: An exemplary use of RPA software
What are some of the RPA use cases in workflows?
Data updates: RPA can automate tasks in data processing workflows. RPA can be set to update relevant data from forms or emails automatically. So, HR, marketing services, and sales departments can have access to the most up-to-date and correct information.
Data validation: Data validation workflows can include automatable tasks like cross-checking data against publicly available data. RPA automation is more suitable than other tools due to scalability.
Data extraction from PDFs and scanned documents: RPA can automate tasks in data extraction document processing because screen scraping, OCR (Optical Character Recognition), and basic pattern recognition technologies can be integrated with RPAs.
Learn more about RPA use cases and applications.
You can download our whitepaper to find out more about workload automation:
If you have any more questions about best practices for workload automation, please get in touch with us at:
Cem regularly speaks at international technology conferences. He graduated from Bogazici University as a computer engineer and holds an MBA from Columbia Business School.
YOUR EMAIL ADDRESS WILL NOT BE PUBLISHED. REQUIRED FIELDS ARE MARKED
*
0 CommentsComment
Is Bing Image Creator Free?
Bing Image Creator is a remarkable tool that harnesses the power of artificial intelligence to help users design customized images. By simply inputting a prompt or a keyword, you can generate unique images that align with your creative vision. Whether you need images for your blog posts, social media campaigns, or website visuals, Bing Image Creator offers a seamless and efficient solution.
Yes, Bing Image Creator is indeed a free tool that allows users to generate custom images using artificial intelligence. Users can input a prompt or a keyword, and the tool will generate an image based on that input. The generated image can be further customized using various settings such as color, style, and layout. Users have the flexibility to create as many images as they want without incurring any charges. Additionally, the images created with Bing Image Creator can be used for personal or commercial purposes. While users have the option to pay for additional boosts if they run out, the core functionality of Bing Image Creator remains free to use.
Also read: How to Use Bing AI Image Generator?
Using Bing Image Creator is incredibly simple and intuitive. Just follow these steps:
Visit the Bing Image Creator website.
Enter a prompt or keyword related to the image you want to create.
Explore the generated image options.
Customize the image using various settings such as color, style, and layout.
Download the image in high-quality format.
With its powerful AI algorithms, Bing Image Creator ensures that the generated images are relevant, visually appealing, and tailored to your specifications.
Bing Image Creator provides users with an extensive range of customization options to enhance their designs. You can choose from a vast library of templates, fonts, icons, and stickers to personalize your images. Additionally, the tool allows you to upload your own photos and apply filters, effects, and text overlays to create a truly unique visual experience. With Bing Image Creator, you have the freedom to express your creativity and create visuals that leave a lasting impression.
Also read: Microsoft adds Voice Chat to AI-Powered Bing Chat on Desktop
Bing Image Creator offers a wide range of image design possibilities to suit your diverse needs. Whether you’re looking to make collages, memes, logos, flyers, posters, or any other visual content, this tool has got you covered. Its flexible features and intuitive interface enable you to experiment with different styles, layouts, and formats, allowing you to bring your ideas to life effortlessly. With Bing Image Creator, you have the power to design visuals that resonate with your target audience.
Once you’ve created your masterpiece with Bing Image Creator, sharing it with the world is a breeze. You can directly share your images to popular platforms such as social media networks, blogs, or websites. Furthermore, the tool provides an easy embed option, allowing you to showcase your images on your own website or blog seamlessly. This ensures that your creations reach a wider audience and make a lasting impact.
Are you ready to take your image design skills to the next level? Bing Image Creator empowers you to unleash your creativity and design visually stunning images that stand out from the crowd. Express yourself, promote your brand, and share your ideas with confidence. With its AI-powered capabilities, user-friendly interface, and vast customization options, Bing Image Creator is the ultimate tool for image enthusiasts and professionals alike.
In conclusion, Bing Image Creator is a remarkable free online tool that empowers users to design captivating images effortlessly. With its AI-driven technology, extensive customization options, and seamless sharing capabilities, this tool opens up a world of creative possibilities. Whether you’re an aspiring blogger, a social media influencer, or a business owner, Bing Image Creator is your go-to resource for visually engaging content. Start using Bing Image Creator today and unlock the true potential of your creativity.
Is Bing Image Creator free to use? Yes, Bing Image Creator is free to use. You can create as many images as you want without any charges.
Can I customize the images generated by Bing Image Creator? Absolutely! Bing Image Creator provides extensive customization options, allowing you to personalize the images according to your preferences.
Can I use the images created with Bing Image Creator for commercial purposes? Yes, you have the freedom to use the images you create with Bing Image Creator for both personal and commercial purposes.
Are there any limitations on the number of images I can create? No, there are no limitations. You can create as many images as you desire using Bing Image Creator.
Where can I share or embed the images created with Bing Image Creator? You can share your images directly to various platforms, including social media networks, blogs, and websites. Bing Image Creator also provides an easy embed option for showcasing images on your own website or blog.
Share this:
Like
Loading…
Related
Automated Intent Classification Using Deep Learning In Google Sheets
We also learned how to automatically populate Google Sheets in Python.
Wouldn’t it be cool if we could perform our intent classification directly in Google Sheets?
That is exactly what we will do here!
Introducing Google Apps ScriptOne limitation of the built-in functions in Google Sheets is that it limits you to predefined behavior.
The good news is that you can define custom functions with new behavior if you can code them yourself in Google Apps Script.
Google Apps Script is based on JavaScript and adds additional functionality that helps interact with Sheets, Docs and other Google Apps.
We are going to define a new custom function named fetchPrediction that will take keywords in Google Sheet cells, and run them through a BERT-powered predictive model to get the intention of search users.
Here is our plan of action:
Learn to review and update values in Google Sheets from Apps Script.
Practice fetching results from an API and populate a sheet with the retrieved values.
Train our BERT-powered predictive model using Uber’s Ludwig.
Use Ludwig to power an API we can call from Apps Script.
Learn some new tools and concepts that help us connect both services together.
Let’s get started!
Retrieving Keyword Data From Google SheetsThis is an empty Google sheet with some barcode related keywords we pulled from SEMrush.
In our first exercise, we will read and print the first 10 keywords from column A.
This is a built-in IDE (Integrated Development Environment) for Google Sheets.
We are going to write a simple JavaScript function called logKeywords that will read all the keywords in our sheet and log them to the console.
Please refer to the official documentation here.
function logKeywords() { var data = sheet.getDataRange().getValues(); for (var i = 0; i < data.length; i++) { console.log('Keyword: ' + data[i][0]); } }Let’s walk over the function, step by step.
We first get a reference to the active sheet, in this case, it is Sheet1.
We didn’t need to authenticate.
It is a good idea to keep this page in another tab as you will refer to it often as your code and want to see if the changes worked.
Now, we printed more than 100 rows, which took a bit of time. When you are writing and testing your code, it is better to work with smaller lists.
We can make a simple change in the loop to fix that.
function logKeywords() { var data = sheet.getDataRange().getValues(); for (var i = 0; i < 10; i++) { console.log('Keyword: ' + data[i][0]); } }When you run this, it not only runs faster but checking the log is also a lot faster.
Add a Column with keyword IDsNext, let’s learn to add data to the sheet.
We are going to write a new function named addIDtoKeywords. It creates a column with one numeric ID per keyword.
There isn’t a lot of value in doing this, but it should help you test the technique with something super simple.
Here is the code to do that.
function addIDtoKeywords() { var data = sheet.getRange("B1"); var values = []; length = 100; for (var i = 1; i <= length+1; i++){ values.push([i]); } console.log(values.length); var column = sheet.getRange("B2:B102"); column.setValues(values); }You should get a new column in the sheet with numbers in increasing order.
We can also add a column header in bold named Keyword ID using the following code.
data.setValue("Keyword ID"); data.setFontWeight("bold");This is what the updated output looks like.
It is a very similar code. Let’s review the changes.
I added a JavaScript array named values to hold the keyword IDs.
During the loop, I added a line to add each ID generated within the loop to the array.
values.push([i]);I printed the length of the value array at the end of the loop to make sure the correct number of IDs was generated.
Finally, I need to get the values to the sheet.
var column = sheet.getRange("B2:B102");This code selects the correct cells to populate and then I can simply set their value using the list I generated.
column.setValues(values);It can’t get simpler than this!
Fetching API Results From Apps ScriptIn the next exercise, we will learn to perform API requests from Apps Script.
We are going to adapt code from step 11 which pulls data from a Books API.
Instead of fetching books, we will translate keywords using the Google Translate API.
Now, we are starting to write more useful code!
Here is a new function named fetchTranslation based on code adapted from step 11.
function fetchTranslation(TEXT){ API_KEY="INPUT YOUR API KEY"; var response = UrlFetchApp.fetch(url, {'muteHttpExceptions': true}); var json = response.getContentText(); translation = JSON.parse(json); return translation["data"]["translations"][0]["translatedText"]; }This function takes an input text, encodes it and inserts it into an API URL to call the Google Translate service.
There is an API key we need to get and also we need to enable to Translate service. I also recommend restricting the API to the IP you are using to test during development.
Once we have the API URL to call, it is as simple as calling this code.
var response = UrlFetchApp.fetch(url, {'muteHttpExceptions': true});The next lines get us the response in JSON format and after a bit of navigation down the JSON tree, we get the translated text.
As you can see in my code, I like to log almost every step in the code to the console to confirm it is doing what I expect.
Here is one example of how I figured out the correct JSON path sequence.
You can see the progression in the logs here, including the final output.
Translating KeywordsAs we tested the function and it works, we can proceed to create another function to fetch and translate the keywords from the sheet.
We will build up from what we’ve learned so far.
We will call this function a super original name TranslateKeywords!
function TranslateKeywords() { var header = sheet.getRange("B1"); header.setValue("Translation"); header.setFontWeight("bold"); var keyword = sheet.getRange("A2").getValue(); console.log(keyword); translated_keyword = fetchTranslation(keyword); console.log(translated_keyword); var data = sheet.getRange("B2"); data.setValue(translated_keyword); }The code in this function is very similar to the one we used to set Keyword IDs.
The main difference is that we pass the keyword to our new fetchTranslation function and update a single cell with the result.
Here is what it looks like for our example keyword.
As you can probably see, there is no for loop, so this will only update one single row/keyword. The first one.
Please complete the for loop to get the translation for all keywords as a homework exercise.
Building an Intent Classification ModelLet’s move to build our intent classification service that we will call to populate keyword intents.
In my previous deep learning articles, I’ve covered Ludwig, Uber’s AI toolbox.
I like it a lot because it allows you to build state-of-the-art deep learning models without writing a single line of code.
It is also very convenient to run in Google Colab.
We are going to follow the same steps I described in this article, this will give us a powerful intent prediction model powered by BERT.
Here is a quick summary of the steps you need paste into Google Colab (make sure to select the GPU runtime!).
Please refer to my article for the context:
%tensorflow_version 1.x import tensorflow as tf; print(tf.__version__) !pip install ludwig #upload Question_Classification_Dataset.csv and 'Question Report_Page 1_Table.csv' from google.colab import files files.upload() import pandas as pd df = pd.read_csv("Question_Classification_Dataset.csv", index_col=0) !unzip uncased_L-12_H-768_A-12.zip # create the ludwig configuration file for BERT-powered classification template=""" input_features: - name: Questions type: text encoder: bert config_path: uncased_L-12_H-768_A-12/bert_config.json checkpoint_path: uncased_L-12_H-768_A-12/bert_model.ckpt preprocessing: word_tokenizer: bert word_vocab_file: uncased_L-12_H-768_A-12/vocab.txt padding_symbol: '[PAD]' unknown_symbol: '[UNK]' output_features: - name: Category0 type: category - name: Category2 type: category text: word_sequence_length_limit: 128 training: batch_size: 32 learning_rate: 0.00002 """ with open("model_definition.yaml", "w") as f: f.write(template) !pip install bert-tensorflow !ludwig experiment --data_csv Question_Classification_Dataset.csv --model_definition_file model_definition.yamlAfter completing these steps in Google Colab, we should get a high accuracy predictive model for search intent.
We can verify the predictions with this code.
test_df = pd.read_csv("Question Report_Page 1_Table.csv") #we rename Query to Questions to match what the model expects predictions = model.predict(test_df.rename(columns={'Query': 'Questions'} )) test_df.join(predictions)[["Query", "Category2_predictions"]]We get a data frame like this one.
The intentions predicted are not the ones you typically expect: navigational, transactional, informational, but they are good enough to illustrate the concept.
Please check an awesome article by Kristin Tynski that explains how to expand this concept to get true search intents.
Turning Our Model Into an API ServiceLudwig has one super cool feature that allows you to serve models directly as an API service.
The command for this is Ludwig serve.
I was trying to accomplish the same thing following a super complicated path because I didn’t check that something like this already existed. 🤦
It is not installed by default, we need to install it with this command.
!pip install ludwig[serve]We can check the command-line options with:
!ludwig serve --helpCreating an API from our model is as simple as running this command.
!ludwig serve -m results/experiment_run/model INFO: Started server process [5604] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Shutting down INFO: Finished server process [5604]As we are running this code in the notebook, we need to use a little trick to push this process to the background (a separate thread).
%%bash --bgThe magic command %%bash –bg runs the shellcode in a separate thread returning control to the notebook so we can run code that can interact with the service.
I found this to be a super cool and valuable trick. I’m also introducing more shell tricks that I learned many years ago.
The nohup command prevents the process from getting killed when the parent dies. It is optional here.
We can track the progress of the background process using this command.
!tail debug.logAfter you see this message, you can proceed to the next step.
Let’s send a test API request using curl to see if the service works.
You should get this response back.
{"Category0_predictions":"HUMAN","Category0_probabilities_":0.00021219381596893072,"Category0_probabilities_ENTITY":7.17515722499229e-05,"Category0_probabilities_HUMAN":0.9988889098167419,"Category0_probabilities_DESCRIPTION":0.000423480843892321,"Category0_probabilities_NUMERIC":2.7793401386588812e-05,"Category0_probabilities_LOCATION":0.0003020864969585091,"Category0_probabilities_ABBREVIATION":7.374086999334395e-05,"Category0_probability":0.9988889098167419,"Category2_predictions":"ind","Category2_probabilities_":8.839580550557002e-05,"Category2_probabilities_ind":0.9759176969528198,"Category2_probabilities_other":0.0013697665417566895,"Category2_probabilities_def":3.929347076336853e-05,"Category2_probabilities_count":4.732362140202895e-05,"Category2_probabilities_desc":0.014149238355457783,"Category2_probabilities_manner":7.225596345961094e-05,"Category2_probabilities_date":7.537546480307356e-05,"Category2_probabilities_cremat":0.00012272763706278056,"Category2_probabilities_reason":0.00042629052768461406,"Category2_probabilities_gr":0.0025540771894156933,"Category2_probabilities_country":0.0002626778441481292,"Category2_probabilities_city":0.0004305317997932434,"Category2_probabilities_animal":0.00024954770924523473,"Category2_probabilities_food":8.139225974446163e-05,"Category2_probabilities_dismed":7.852958515286446e-05,"Category2_probabilities_termeq":0.00023714809503871948,"Category2_probabilities_period":4.197505040792748e-05,"Category2_probabilities_money":3.626687248470262e-05,"Category2_probabilities_exp":5.991378566250205e-05,"Category2_probabilities_state":0.00010361814202342297,"Category2_probabilities_sport":8.741072088014334e-05,"Category2_probabilities_event":0.00013374585250858217,"Category2_probabilities_product":5.6306344049517065e-05,"Category2_probabilities_substance":0.00016623239207547158,"Category2_probabilities_color":1.9601659005274996e-05,"Category2_probabilities_techmeth":4.74867774755694e-05,"Category2_probabilities_dist":9.92789282463491e-05,"Category2_probabilities_perc":3.87108520953916e-05,"Category2_probabilities_veh":0.00011915313370991498,"Category2_probabilities_word":0.00016430433606728911,"Category2_probabilities_title":0.0010781479068100452,"Category2_probabilities_mount":0.00024070330255199224,"Category2_probabilities_body":0.0001515906333224848,"Category2_probabilities_abb":8.521509153069928e-05,"Category2_probabilities_lang":0.00022924368386156857,"Category2_probabilities_plant":4.893113509751856e-05,"Category2_probabilities_volsize":0.0001462997024646029,"Category2_probabilities_symbol":9.98345494735986e-05,"Category2_probabilities_weight":8.899033855414018e-05,"Category2_probabilities_instru":2.636547105794307e-05,"Category2_probabilities_letter":3.7610192521242425e-05,"Category2_probabilities_speed":4.142118996242061e-05,"Category2_probabilities_code":5.926147059653886e-05,"Category2_probabilities_temp":3.687662319862284e-05,"Category2_probabilities_ord":6.72415699227713e-05,"Category2_probabilities_religion":0.00012743560364469886,"Category2_probabilities_currency":5.8569487009663135e-05,"Category2_probability":0.9759176969528198} Exposing Our Service Using NgrokSo, we have a new API that can make intent predictions, but one big problem is that it is only accessible from within our Colab notebook.
Let me introduce another cool service that I use often, Ngrok.
Ngrok helps you create publicly accessible URLs that connect to a local service like the one we just created.
I do not recommend doing this for production use, but it is very handy during development and testing.
You don’t need to create an account, but I personally do it because I get to set up a custom subdomain that I use very frequently.
Here are the steps to give our API a public URL to call from App Script.
We first download and uncompress ngrok.
%%bash --bgThe code above tells ngrok to connect to the local service in port 8000. That is all we need to do.
You can confirm it works by repeating the curl call, but calling the public URL. You should get the same result.
If you don’t want to set up a custom domain, you can use this code instead.
%%bash --bgThis will generate a random public URL and you get retrieve with this code.
"import sys, json; print(json.load(sys.stdin)['tunnels'][0]['public_url'])"
Now, we get back to our final steps.
Fetching Intent PredictionsWe are going to adapt the code we used to make Google Translate API requests so we can make intent prediction requests.
One big difference between the two API services is that we need to make HTTP POST requests instead of simpler HTTP GET requests.
Let’s see how that changes our code and learn a bit more about HTTP in the process.
function fetchPrediction(question = "who is the boss?"){ TEXT = encodeURI(TEXT); console.log(TEXT); var options = { "method" : "POST", "contentType" : "application/x-www-form-urlencoded", "payload" : TEXT, 'muteHttpExceptions': true }; var response = UrlFetchApp.fetch(url, options); var json = response.getContentText(); prediction = JSON.parse(json); console.log(prediction["Category0_predictions"]); return prediction["Category0_predictions"]; }The function fetchPrediction calls the API service we created and returns the predicted intent. It basically reproduces the equivalent of the curl call we made Colab, but in Apps Script.
I highlighted some key changes in the code. Let’s review them.
One key difference between GET and POST requests is that in GET requests the data is passed in the URL as parameters.
In POST requests, the data is passed inside the body of the request.
We need to format the data before we pass it in the body and we need to set the correct content type so the server knows how to decode it.
This line encodes the question we are passing.
TEXT = encodeURI(TEXT);This is an example of what the encoded TEXT looks like.
Questions=label%20generatorThe correct content type for this encoding is application/x-www-form-urlencoded. This is recommended encoding for HTML form data.
We create an options data structure where we specify these settings and the correct request type and we are set to go.
You should see the encoded input and predicted intent in the logs.
How do we get the intentions for all the keywords in the sheet?
You might be thinking we will create another function that will read the keywords in a loop and populate the intentions. Not at all!
We can simply call this function by name directly from the sheet! How cool is that?
Resources to Learn MoreCombining simple App Script functions with powerful API backends that you can code in any language opens the doors to infinite productivity hacks.
Here are some of the resources I read while putting this together.
Finally, let me highlight a very important and valuable project that JR Oakes started.
It is an awesome repository for Python and JavaScript projects from the coders in the SEO community. I plan to find time to upload my code snippets, please make sure to contribute yours.
For some reason, this non-issue keeps popping up in my Twitter feed. I will leave this tweet here as a friendly reminder. ✌️
— Hamlet 🇩🇴 🇺🇸 (@hamletbatista) March 10, 2023
More Resources:
Image Credits
All screenshots taken by author, March 2023
Automated Mistake By Apple Kills All Mac Developer’s Apps
Developer Charlie Monroe, creator of the Downie video downloader, among other apps, said that Apple didn’t even send him a message saying it had happened, and for several hours he didn’t know whether he still had a business or not…
Monroe described the experience in a blog post:
On Aug 4, 2023, I woke up to a slightly different world — I had lost my business as it seemed. Full inbox of reports of my apps not launching (crashing on launch) and after not too long I found out that when I sign into my Apple developer account I can no longer see that I would be enrolled into Apple’s developer program […]
After more investigation, I found out that the distribution certificates were revoked — evidently by Apple as no one else has access to them and I was sound asleep when all this happened. Each macOS app these days needs to be co-designed using an Apple-issued certificate so that the app will flawlessly work on all computers. When Apple revokes the certificate, it’s generally a remove kill-switch for the apps.
I got really frightened as all of sudden, no user was able to use my apps anymore […] As it was 7 a.m. (all times are CET), Apple’s contact form only showed the option to send them an email — so I did. At 9 a.m. with my teeth grinding, I went for the phone option where you leave a number and they call you back. Didn’t.
At this point you no longer know whether you have a business or not. Should I quickly go and apply for a job? Or should I try to found another company and distribute the apps under it? What should I do?
He said one of the most alarming aspects of it was the damage to his reputation.
The most damaging to me is the message shown to user:
I really find the above borderlining on slander.
This was echoed by a Downie user.
Hi. I want to let you know that I spent two and a half hours on the phone with @Apple trying to get them to say exactly how Downie (change the name) will harm my computer. They said it was malicious code detected. If that was an error, your reputation has absolutely been harmed.
— chúng tôi (@JTWilliams_me) August 5, 2023
He said that it took Apple 24 hours to partly fix the problem, removing the flags, though that still left him having to recompile, re-sign, and redistribute everything. This was initially done without any contact from Apple.
Apple did later call back, explaining that his account was “erroneously flagged by automated processes as malicious and was put on hold.”
It seems incredible that all this could happen without human intervention. Apple does, of course, have to act swiftly when there is a chance of malware in the Mac App Store, but you would have thought it would have pinged a human being to verify the situation before inconveniencing significant number of Mac users, and potentially doing permanent damage to a developer’s reputation. Most app users will never know the story behind this, only that they bought an app, Apple told them it was malware, and they deleted it as instructed.
It also seems unlikely to help Apple’s antitrust battles, where many are arguing that the company holds too much power over users and developers alike.
FTC: We use income earning auto affiliate links. More.
10 Automated Machine Learning For Supervised Learning (Part 2)
This article was published as a part of the Data Science Blogathon
IntroductionThis post will discuss 10 Automated Machine Learning (autoML) packages that we can run in Python. If you are tired of running lots of Machine Learning algorithms just to find the best one, this post might be what you are looking for. This post is the second part of this first post. The first part explains the general concept of Machine Learning from defining the objective, pre-processing, model creation and selection, hyperparameter-tuning, and model evaluation. At the end of that post, Auto-Sklearn is introduced as an autoML. If you are already familiar with Machine Learning, you can skip that part 1.
The main point of the first part is that we require a relatively long time and many lines of code to run all of the classification or regression algorithms before finally selecting or ensembling the best models. Instead, we can run AutoML to automatically search for the best model with a much shorter time and, of course, less code. Please find my notebooks for conventional Machine Learning algorithms for regression (predicting house prices) and classification (predicting poverty level classes) tasks in the table below. My AutoML notebooks are also in the table below. Note that this post will focus only on regression and classification AutoML while AutoML also can be applied for image, NLP or text, and time series forecasting.
Now, we will start discussing the 10 AutoML packages that can replace those long notebooks.
AutoSklearnThis autoML, as mentioned above, has been discussed before. Let’s do a quick recap. Below is the code for searching for the best model in 3 minutes. More details are explained in the first part.
Regression
!apt install -y build-essential swig curl !pip install auto-sklearn !pip install scipy==1.7.0 !pip install -U scikit-learn from autosklearn.regression import AutoSklearnRegressor from sklearn.metrics import mean_squared_error, mean_absolute_errorRunning AutoSklearn
# Create the model sklearn = AutoSklearnRegressor(time_left_for_this_task=3*60, per_run_time_limit=30, n_jobs=-1) # Fit the training data sklearn.fit(X_train, y_train) # Sprint Statistics print(sklearn.sprint_statistics()) # Predict the validation data pred_sklearn = sklearn.predict(X_val) # Compute the RMSE rmse_sklearn=MSE(y_val, pred_sklearn)**0.5 print('RMSE: ' + str(rmse_sklearn))Output:
auto-sklearn results: Dataset name: c40b5794-fa4a-11eb-8116-0242ac130202 Metric: r2 Best validation score: 0.888788 Number of target algorithm runs: 37 Number of successful target algorithm runs: 23 Number of crashed target algorithm runs: 0 Number of target algorithms that exceeded the time limit: 8 Number of target algorithms that exceeded the memory limit: 6 RMSE: 27437.715258009852Result:
# Scatter plot true and predicted values plt.scatter(pred_sklearn, y_val, alpha=0.2) plt.xlabel('predicted') plt.ylabel('true value') plt.text(100000, 400000, 'RMSE: ' + str(round(rmse_sklearn))) plt.text(100000, 350000, 'MAE: ' + str(round(mean_absolute_error(y_val, pred_sklearn)))) plt.text(100000, 300000, 'R: ' + str(round(np.corrcoef(y_val, pred_sklearn)[0,1],4))) plt.show()Fig. 1 AutoSklearn Regression Result
Classification
Applying AutoML
!apt install -y build-essential swig curl from autosklearn.classification import AutoSklearnClassifier # Create the model sklearn = AutoSklearnClassifier(time_left_for_this_task=3*60, per_run_time_limit=15, n_jobs=-1) # Fit the training data sklearn.fit(X_train, y_train) # Sprint Statistics print(sklearn.sprint_statistics()) # Predict the validation data pred_sklearn = sklearn.predict(X_val) # Compute the accuracy print('Accuracy: ' + str(accuracy_score(y_val, pred_sklearn)))Output:
auto-sklearn results: Dataset name: 576d4f50-c85b-11eb-802c-0242ac130202 Metric: accuracy Best validation score: 0.917922 Number of target algorithm runs: 40 Number of successful target algorithm runs: 8 Number of crashed target algorithm runs: 0 Number of target algorithms that exceeded the time limit: 28 Number of target algorithms that exceeded the memory limit: 4 Accuracy: 0.923600209314495Result:
# Prediction results print('Confusion Matrix') print(pd.DataFrame(confusion_matrix(y_val, pred_sklearn), index=[1,2,3,4], columns=[1,2,3,4])) print('') print('Classification Report') print(classification_report(y_val, pred_sklearn))Output:
Confusion Matrix 1 2 3 4 1 123 14 3 11 2 17 273 11 18 3 3 17 195 27 4 5 15 5 1174 Classification Report precision recall f1-score support 1 0.83 0.81 0.82 151 2 0.86 0.86 0.86 319 3 0.91 0.81 0.86 242 4 0.95 0.98 0.97 1199 accuracy 0.92 1911 macro avg 0.89 0.86 0.88 1911 weighted avg 0.92 0.92 0.92 1911 Tree-based Pipeline Optimization Tool (TPOT)TPOT is built on top of scikit-learn. TPOT uses a genetic algorithm to search for the best model according to the “generations” and “population size”. The higher the two parameters are set, the longer will it take time. Unlike AutoSklearn, we do not set the specific running time for TPOT. As its name suggests, after the TPOT is run, it exports lines of code containing a pipeline from importing packages, splitting the dataset, creating the tuned model, fitting the model, and finally predicting the validation dataset. The pipeline is exported in .py format.
In the code below, I set the generation and population_size to be 5. The output gives 5 generations with increasing “scoring”. I set the scoring to be “neg_mean_absolute_error” and” accuracy” for regression and classification tasks respectively. Neg_mean_absolute_error means Mean Absolute Error (MAE) in negative form. The algorithm chooses the highest scoring value. Making the MAE negative will make the algorithm selecting the MAE closest to zero.
Regression
from tpot import TPOTRegressor # Create model cv = RepeatedStratifiedKFold(n_splits=3, n_repeats=3, random_state=123) tpot = TPOTRegressor(generations=5, population_size=5, cv=cv, scoring='neg_mean_absolute_error', verbosity=2, random_state=123, n_jobs=-1) # Fit the training data tpot.fit(X_train, y_train) # Export the result tpot.export('tpot_model.py')Output:
Generation 1 - Current best internal CV score: -20390.588131563232 Generation 2 - Current best internal CV score: -19654.82630417806 Generation 3 - Current best internal CV score: -19312.09139004322 Generation 4 - Current best internal CV score: -19312.09139004322 Generation 5 - Current best internal CV score: -18752.921100941825 Best pipeline: RandomForestRegressor(input_matrix, bootstrap=True, max_features=0.25, min_samples_leaf=3, min_samples_split=2, n_estimators=100)Classification
from tpot import TPOTClassifier # TPOT that are stopped earlier. It still gives temporary best pipeline. # Create the model cv = RepeatedStratifiedKFold(n_splits=3, n_repeats=3, random_state=123) tpot = TPOTClassifier(generations=5, population_size=5, cv=cv, scoring='accuracy', verbosity=2, random_state=123, n_jobs=-1) # Fit the training data tpot.fit(X_train, y_train) # Export the result tpot.export('tpot_model.py')Output:
Generation 1 - Current best internal CV score: 0.7432273262661955 Generation 2 - Current best internal CV score: 0.843824979278454 Generation 3 - Current best internal CV score: 0.8545565589146273 Generation 4 - Current best internal CV score: 0.8545565589146273 Generation 5 - Current best internal CV score: 0.859616978580465 Best pipeline: RandomForestClassifier(GradientBoostingClassifier(input_matrix, learning_rate=0.001, max_depth=2, max_features=0.7000000000000001, min_samples_leaf=1, min_samples_split=19, n_estimators=100, subsample=0.15000000000000002), bootstrap=True, criterion=gini, max_features=0.8500000000000001, min_samples_leaf=4, min_samples_split=12, n_estimators=100)AutoSklearn gives the result of RandomForestRegressor for the regression task. As for the classification, it gives the stacking of GradientBoostingClassifier and RandomForestClassifier. All algorithms already have their hyperparameters tuned.
Here is to see the validation data scoring metrics.
Regression
pred_tpot = results # Scatter plot true and predicted values plt.scatter(pred_tpot, y_val, alpha=0.2) plt.xlabel('predicted') plt.ylabel('true value') plt.text(100000, 400000, 'RMSE: ' + str(round(MSE(y_val, pred_tpot)**0.5))) plt.text(100000, 350000, 'MAE: ' + str(round(mean_absolute_error(y_val, pred_tpot)))) plt.text(100000, 300000, 'R: ' + str(round(np.corrcoef(y_val, pred_tpot)[0,1],4))) plt.show()Output:
Fig. 2 TPOT regression result.
Classification
pred_tpot = results # Compute the accuracy print('Accuracy: ' + str(accuracy_score(y_val, pred_tpot))) print('') # Prediction results print('Confusion Matrix') print(pd.DataFrame(confusion_matrix(y_val, pred_tpot), index=[1,2,3,4], columns=[1,2,3,4])) print('') print('Classification Report') print(classification_report(y_val, pred_tpot))Output:
Accuracy: 0.9246467817896389 Confusion Matrix 1 2 3 4 1 117 11 7 16 2 6 288 10 15 3 2 18 186 36 4 5 12 6 1176 Classification Report precision recall f1-score support 1 0.90 0.77 0.83 151 2 0.88 0.90 0.89 319 3 0.89 0.77 0.82 242 4 0.95 0.98 0.96 1199 accuracy 0.92 1911 macro avg 0.90 0.86 0.88 1911 weighted avg 0.92 0.92 0.92 1911 Distributed Asynchronous Hyper-parameter Optimization (Hyperopt)Hyperopt is also usually used to optimize the hyperparameters of one model that has been specified. For example, we decide to apply Random Forest and then run hyperopt to find the optimal hyperparameters for the Random Forest. My previous post discussed it. But, this post is different in that it uses hyperopt to search for the best Machine Learning model automatically, not just tuning the hyperparameters. The code is similar but different.
The code below shows how to use hyperopt to run AutoML. Max evaluation of 50 and trial timeout of 20 seconds is set. These will determine how long the AutoML will work. Like in TPOT, we do not set the time limit in hyperopt.
Regression
from hpsklearn import HyperoptEstimator from hpsklearn import any_regressor from hpsklearn import any_preprocessing from hyperopt import tpe from sklearn.metrics import mean_squared_error # Create the model hyperopt = HyperoptEstimator(regressor=any_regressor(‘reg’), preprocessing=any_preprocessing(‘pre’), loss_fn=mean_squared_error, algo=tpe.suggest, max_evals=50, trial_timeout=20) # Fit the data hyperopt.fit(X_train, y_train)
Classification
from hpsklearn import HyperoptEstimator from hpsklearn import any_classifier from hpsklearn import any_preprocessing from hyperopt import tpe # Create the model hyperopt = HyperoptEstimator(classifier=any_classifier(‘cla’), preprocessing=any_preprocessing(‘pre’), algo=tpe.suggest, max_evals=50, trial_timeout=30) # Fit the training data hyperopt.fit(X_train_ar, y_train_ar)
In the Kaggle notebook (in the table above), every time I finished performing the fitting and predicting, I will show the prediction of validation data results in scatter plot, confusion matrix, and classification report. The code is always almost the same with a little adjustment. So, from this point onwards, I am not going to put them in this post. But, the Kaggle notebook provides them.
To see the algorithm from the AutoML search result. Use the below code. The results are ExtraTreeClassfier and XGBRegressor. Observe that it also searches for the preprocessing technics, such as standard scaler and normalizer.
# Show the models print(hyperopt.best_model())Regression
{'learner': XGBRegressor(base_score=0.5, booster='gbtree', colsample_bylevel=0.6209369845565308, colsample_bynode=1, colsample_bytree=0.6350745975782562, gamma=0.07330922089021298, gpu_id=-1, importance_type='gain', interaction_constraints='', learning_rate=0.0040826994703554555, max_delta_step=0, max_depth=10, min_child_weight=1, missing=nan, monotone_constraints='()', n_estimators=2600, n_jobs=1, num_parallel_tree=1, objective='reg:linear', random_state=3, reg_alpha=0.4669165283261672, reg_lambda=2.2280355282357056, scale_pos_weight=1, seed=3, subsample=0.7295609371405459, tree_method='exact', validate_parameters=1, verbosity=None), 'preprocs': (Normalizer(norm='l1'),), 'ex_preprocs': ()}Classification
{'learner': ExtraTreesClassifier(bootstrap=True, max_features='sqrt', n_estimators=308, n_jobs=1, random_state=1, verbose=False), 'preprocs': (StandardScaler(with_std=False),), 'ex_preprocs': ()} AutoKerasAutoKeras, as you might guess, is an autoML specializing in Deep Learning or Neural networks. The “Keras” in the name gives the clue. AutoKeras helps in finding the best neural network architecture and hyperparameters for the prediction model. Unlike the other AutoML, AutoKeras does not consider tree-based, distance-based, or other Machine Learning algorithms.
Deep Learning is challenging not only for the hyperparameter-tuning but also for the architecture setting. Many ask about how many neurons or layers are the best to use. There is no clear answer to that. Conventionally, users must try to run and evaluate their Deep Learning architectures one by one before finally decide which one is the best. It takes a long time and resources to accomplish. I wrote a post describing this here. However, AutoKeras can solve this problem.
To apply the AutoKeras, I set the max_trials to be 8 and it will try to find the best deep learning architecture for a maximum of 8 trials. Set the epoch while fitting the training dataset and it will determine the accuracy of the model.
Regression
!pip install autokeras import autokeras # Create the model keras = autokeras.StructuredDataRegressor(max_trials=8) # Fit the training dataset keras.fit(X_train, y_train, epochs=100) # Predict the validation data pred_keras = keras.predict(X_val)Classification
!pip install autokeras import autokeras # Create the model keras = autokeras.StructuredDataClassifier(max_trials=8) # Fit the training dataset keras.fit(X_train, y_train, epochs=100) # Predict the validation data pred_keras = keras.predict(X_val) # Compute the accuracy print('Accuracy: ' + str(accuracy_score(y_val, pred_keras)))To find the architecture of the AutoKeras search, use the following code.
# Show the built models keras_export = keras.export_model() keras_export.summary()Regression
Model: "model" Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) [(None, 20)] 0 multi_category_encoding (Mul (None, 20) 0 dense (Dense) (None, 512) 10752 re_lu (ReLU) (None, 512) 0 dense_1 (Dense) (None, 32) 16416 re_lu_1 (ReLU) (None, 32) 0 regression_head_1 (Dense) (None, 1) 33 ================================================================= Total params: 27,201 Trainable params: 27,201 Non-trainable params: 0Classification
Model: "model" Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) [(None, 71)] 0 multi_category_encoding (Mul (None, 71) 0 normalization (Normalization (None, 71) 143 dense (Dense) (None, 512) 36864 re_lu (ReLU) (None, 512) 0 dense_1 (Dense) (None, 32) 16416 re_lu_1 (ReLU) (None, 32) 0 dropout (Dropout) (None, 32) 0 dense_2 (Dense) (None, 4) 132 classification_head_1 (Softm (None, 4) 0 ================================================================= Total params: 53,555 Trainable params: 53,412 Non-trainable params: 143 MLJARMLJAR is another great AutoML. And, you will find why soon. To run MLJAR, I assign the arguments of mode, eval_metric, total_time_limit, and feature_selection. This AutoML will understand whether it is a regression or classification task from the eval_metric. The total_time_limit is the duration of how long we allow MLJAR to run in seconds. In this case, it will take 300 seconds or 5 minutes to find the possible best model. We can also specify whether to allow feature selection. The output then will report the used algorithms and how long they take to finish.
Regression
from supervised.automl import AutoML # Create the model mljar = AutoML(mode="Compete", eval_metric="rmse", total_time_limit=300, features_selection=True) # Fit the training data mljar.fit(X_train, y_train) # Predict the training data mljar_pred = mljar.predict(X_val)Classification
from supervised.automl import AutoML # Create the model mljar = AutoML(mode="Compete", eval_metric="accuracy", total_time_limit=300, features_selection=True) # Fit the training data mljar.fit(X_train, y_train) # Predict the training data mljar_pred = mljar.predict(X_val)
The argument “mode” lets us decide what the MLJAR is expected for. There are 4 types of modes to define the purpose of running the MLJAR. In the example code above, the mode of “compete” is used for winning a competition by finding the best model by tuning and ensembling methods. The mode of “optuna” is used to find the best-tuned model with unlimited computation time. The mode of “perform” builds a Machine Learning pipeline for production. The mode of “explain” is used for data explanation.
The result of MLJAR is automatically reported and visualized. Unfortunately, Kaggle does not display the report result after saving. So, below is how it should look. The report compares the MLJAR results for every algorithm. We can see the ensemble methods have the lowest MSE for the regression task and the highest accuracy for the classification task. The increasing number of iteration lowers the MSE for the regression task and improves the accuracy of the classification task. (The leaderboard tables below actually have more rows, but they were cut.)
Regression
Fig. 3 MLJAR Report for regression
Fig. 4 MLJAR for regession
Fig. 5 MLJAR report for classification (1). Fig. 6 MLJAR report for classification (2).
AutoGluonAutoGluon requires users to format the training dataset using TabularDataset to recognize it. Users can then specify the time_limit allocation for AutoGluon to work. In the example code below, I set it to be 120 seconds or 2 minutes.
Regression
!pip install -U pip !pip install -U setuptools wheel !pip install -U "mxnet<2.0.0" !pip install autogluon from autogluon.tabular import TabularDataset, TabularPredictor # Prepare the data Xy_train = X_train.reset_index(drop=True) Xy_train['Target'] = y_train Xy_val = X_val.reset_index(drop=True) Xy_val['Target'] = y_val X_train_gluon = TabularDataset(Xy_train) X_val_gluon = TabularDataset(Xy_val) # Fit the training data gluon = TabularPredictor(label='Target').fit(X_train_gluon, time_limit=120) # Predict the training data gluon_pred = gluon.predict(X_val)Classification
!pip install -U pip !pip install -U setuptools wheel !pip install -U "mxnet<2.0.0" !pip install autogluon from autogluon.tabular import TabularDataset, TabularPredictor # Prepare the data Xy_train = X_train.reset_index(drop=True) Xy_train['Target'] = y_train Xy_val = X_val.reset_index(drop=True) Xy_val['Target'] = y_val X_train_gluon = TabularDataset(Xy_train) X_val_gluon = TabularDataset(Xy_val) # Fit the training data gluon = TabularPredictor(label='Target').fit(X_train_gluon, time_limit=120) # Predict the training data gluon_pred = gluon.predict(X_val)After finishing the task, AutoGluon can report the accuracy of each Machine Learning algorithm it has tried. The report is called leaderboard. The columns below are actually longer, but I cut them for this post.
# Show the models leaderboard = gluon.leaderboard(X_train_gluon) leaderboardRegression
model score_test score_val pred_time_test . . .
0 RandomForestMSE -15385.131260 -23892.159881 0.133275 . . .
1 ExtraTreesMSE -15537.139720 -24981.601931 0.137063 . . .
2 LightGBMLarge -17049.125557 -26269.841824 0.026560 . . .
3 XGBoost -18142.996982 -23573.451829 0.054067 . . .
4 KNeighborsDist -18418.785860 -41132.826848 0.135036 . . .
5 CatBoost -19585.309377 -23910.403833 0.004854 . . .
6 WeightedEnsemble_L2 -20846.144676 -22060.013365 1.169406 . . .
7 LightGBM -23615.121228 -23205.065207 0.024396 . . .
8 LightGBMXT -25261.893395 -24608.580984 0.015091 . . .
9 NeuralNetMXNet -28904.712029 -24104.217749 0.819149 . . .
10 KNeighborsUnif -39243.784302 -39545.869493 0.132839 . . .
11 NeuralNetFastAI -197411.475391 -191261.448480 0.070965 . . .
Classification
model score_test score_val pred_time_test . . .
0 WeightedEnsemble_L2 0.986651 0.963399 3.470253 . . .
1 LightGBM 0.985997 0.958170 0.600316 . . .
2 XGBoost 0.985997 0.956863 0.920570 . . .
3 RandomForestEntr 0.985866 0.954248 0.366476 . . .
4 RandomForestGini 0.985735 0.952941 0.397669 . . .
5 ExtraTreesEntr 0.985735 0.952941 0.398659 . . .
6 ExtraTreesGini 0.985735 0.952941 0.408386 . . .
7 KNeighborsDist 0.985473 0.950327 2.013774 . . .
8 LightGBMXT 0.984034 0.951634 0.683871 . . .
9 NeuralNetFastAI 0.983379 0.947712 0.340936 . . .
10 NeuralNetMXNet 0.982332 0.956863 2.459954 . . .
11 CatBoost 0.976574 0.934641 0.044412 . . .
12 KNeighborsUnif 0.881560 0.769935 1.970972 . . .
13 LightGBMLarge 0.627143 0.627451 0.014708 . . .
H2OSimilar to AutoGluon, H2O requires the training dataset in a certain format, called H2OFrame. To decide how long H2O will work, either max_runtime_secs or max_models must be specified. The names explain what they mean.
Regression
import h2o from h2o.automl import H2OAutoML h2o.init() # Prepare the data Xy_train = X_train.reset_index(drop=True) Xy_train['SalePrice'] = y_train.reset_index(drop=True) Xy_val = X_val.reset_index(drop=True) Xy_val['SalePrice'] = y_val.reset_index(drop=True) # Convert H2O Frame Xy_train_h2o = h2o.H2OFrame(Xy_train) X_val_h2o = h2o.H2OFrame(X_val) # Create the model h2o_model = H2OAutoML(max_runtime_secs=120, seed=123) # Fit the model h2o_model.train(x=Xy_train_h2o.columns, y='SalePrice', training_frame=Xy_train_h2o) # Predict the training data h2o_pred = h2o_model.predict(X_val_h2o)Classification
import h2o from h2o.automl import H2OAutoML h2o.init() # Convert H2O Frame Xy_train_h2o = h2o.H2OFrame(Xy_train) X_val_h2o = h2o.H2OFrame(X_val) Xy_train_h2o['Target'] = Xy_train_h2o['Target'].asfactor() # Create the model h2o_model = H2OAutoML(max_runtime_secs=120, seed=123) # Fit the model h2o_model.train(x=Xy_train_h2o.columns, y='Target', training_frame=Xy_train_h2o) # Predict the training data h2o_pred = h2o_model.predict(X_val_h2o) h2o_predFor the classification task, the prediction result is a multilabel classification result. It gives the probability value for each class. Below is an example of the classification.
predict p1 p2 p3 p4
4 0.0078267 0.0217498 0.0175197 0.952904
4 0.00190617 0.00130162 0.00116375 0.995628
4 0.00548938 0.0156449 0.00867845 0.970187
3 0.00484961 0.0161661 0.970052 0.00893224
2 0.0283297 0.837641 0.0575789 0.0764503
3 0.00141621 0.0022694 0.992301 0.00401299
4 0.00805432 0.0300103 0.0551097 0.906826
H2O reports its result by a simple table showing various scoring metrics of each Machine Learning algorithm.
# Show the model results leaderboard_h2o = h2o.automl.get_leaderboard(h2o_model, extra_columns = 'ALL') leaderboard_h2oRegression output:
model_id mean_residual_deviance rmse mse mae rmsle …
GBM_grid__1_AutoML_20230811_022746_model_17 8.34855e+08 28893.9 8.34855e+08 18395.4 0.154829 …
GBM_1_AutoML_20230811_022746 8.44991e+08 29068.7 8.44991e+08 17954.1 0.149824 …
StackedEnsemble_BestOfFamily_AutoML_20230811_022746 8.53226e+08 29210 8.53226e+08 18046.8 0.149974 …
GBM_grid__1_AutoML_20230811_022746_model_1 8.58066e+08 29292.8 8.58066e+08 17961.7 0.153238 …
GBM_grid__1_AutoML_20230811_022746_model_2 8.91964e+08 29865.8 8.91964e+08 17871.9 0.1504 …
GBM_grid__1_AutoML_20230811_022746_model_10 9.11731e+08 30194.9 9.11731e+08 18342.2 0.153421 …
GBM_grid__1_AutoML_20230811_022746_model_21 9.21185e+08 30351 9.21185e+08 18493.5 0.15413 …
GBM_grid__1_AutoML_20230811_022746_model_8 9.22497e+08 30372.6 9.22497e+08 19124 0.159135 …
GBM_grid__1_AutoML_20230811_022746_model_23 9.22655e+08 30375.2 9.22655e+08 17876.6 0.150722 …
XGBoost_3_AutoML_20230811_022746 9.31315e+08 30517.5 9.31315e+08 19171.1 0.157819 …
Classification
model_id mean_per_class_error logloss rmse mse …
StackedEnsemble_BestOfFamily_AutoML_20230608_143533 0.187252 0.330471 0.309248 0.0956343 …
StackedEnsemble_AllModels_AutoML_20230608_143533 0.187268 0.331742 0.309836 0.0959986 …
DRF_1_AutoML_20230608_143533 0.214386 4.05288 0.376788 0.141969 …
GBM_grid__1_AutoML_20230608_143533_model_1 0.266931 0.528616 0.415268 0.172447 …
XGBoost_grid__1_AutoML_20230608_143533_model_1 0.323726 0.511452 0.409528 0.167713 …
GBM_4_AutoML_20230608_143533 0.368778 1.05257 0.645823 0.417088 …
GBM_grid__1_AutoML_20230608_143533_model_2 0.434227 1.10232 0.663382 0.440075 …
GBM_3_AutoML_20230608_143533 0.461059 1.08184 0.655701 0.429944 …
GBM_2_AutoML_20230608_143533 0.481588 1.08175 0.654895 0.428887 …
XGBoost_1_AutoML_20230608_143533 0.487381 1.05534 0.645005 0.416031 …
PyCaretThis is the longest AutoML code that this post is exploring. PyCaret does not need splitting the features (X_train) and label (y_train). So, the below code will only randomly split the training dataset into another training dataset and a validation dataset. Preprocessing, such as filling missing data or feature selection, is also not required. Then, we set up the PyCaret by assigning the data, target variable or label, numeric imputation method, categorical imputation method, whether to use normalization, whether to remove multicollinearity, etc.
Regression
!pip install pycaret from pycaret.regression import * # Generate random numbers val_index = np.random.choice(range(trainSet.shape[0]), round(trainSet.shape[0]*0.2), replace=False) # Split trainSet trainSet1 = trainSet.drop(val_index) trainSet2 = trainSet.iloc[val_index,:] # Create the model caret = setup(data = trainSet1, target='SalePrice', session_id=111, numeric_imputation='mean', categorical_imputation='constant', normalize = True, combine_rare_levels = True, rare_level_threshold = 0.05, remove_multicollinearity = True, multicollinearity_threshold = 0.95)Classification
!pip install pycaret from pycaret.classification import * # Generate random numbers val_index = np.random.choice(range(trainSet.shape[0]), round(trainSet.shape[0]*0.2), replace=False) # Split trainSet trainSet1 = trainSet.drop(val_index) trainSet2 = trainSet.iloc[val_index,:] # Create the model caret = setup(data = trainSet1, target='Target', session_id=123, numeric_imputation='mean', categorical_imputation='constant', normalize = True, combine_rare_levels = True, rare_level_threshold = 0.05, remove_multicollinearity = True, multicollinearity_threshold = 0.95)After that, we can run the PyCaret by specifying how many cross-validations folds we want. The PyCaret for regression gives several models sorting from the best scoring metrics. The top models are Bayesian Ridge, Huber Regressor, Orthogonal Matching Pursuit, Ridge Regression, and Passive-Aggressive Regressor. The scoring metrics are MAE, MSE, RMSE, R2, RMSLE, and MAPE. PyCaret for classification also gives several models. The top models are the Extra Trees Classifier, Random Forest Classifier, Decision Tree Classifier, Extreme Gradient Boosting, and Light Gradient Boosting Machine. The below tables are limited for their rows and columns. Find the complete tables in the Kaggle notebook.
# Show the models caret_models = compare_models(fold=5)Regression
Model MAE MSE RMSE R2 …
br Bayesian Ridge 15940.2956 566705805.8954 23655.0027 0.9059 …
huber Huber Regressor 15204.0960 588342119.6640 23988.3772 0.9033 …
omp Orthogonal Matching Pursuit 16603.0485 599383228.9339 24383.2437 0.9001 …
ridge Ridge Regression 16743.4660 605693331.2000 24543.6840 0.8984 …
par Passive Aggressive Regressor 15629.1539 630122079.3113 24684.8617 0.8972 …
… … … … … … …
Classification
Model Accuracy AUC Recall Prec. …
et Extra Trees Classifier 0.8944 0.9708 0.7912 0.8972 …
rf Random Forest Classifier 0.8634 0.9599 0.7271 0.8709 …
dt Decision Tree Classifier 0.8436 0.8689 0.7724 0.8448 …
xgboost Extreme Gradient Boosting 0.8417 0.9455 0.7098 0.8368 …
lightgbm Light Gradient Boosting Machine 0.8337 0.9433 0.6929 0.8294 …
… … … … … … …
To create the top 5 models, run the following code.
Regression
# Create the top 5 models br = create_model('br', fold=5) huber = create_model('huber', fold=5) omp = create_model('omp', fold=5) ridge = create_model('ridge', fold=5) par = create_model('par', fold=5)Classification
# Create the top 5 models et = create_model('et', fold=5) rf = create_model('rf', fold=5) dt = create_model('dt', fold=5) xgboost = create_model('xgboost', fold=5) lightgbm = create_model('lightgbm', fold=5)To tune the selected model, run the following code.
Regression
# Tune the models, BR: Regression br_tune = tune_model(br, fold=5) # Show the tuned hyperparameters, for example for BR: Regression plot_model(br_tune, plot='parameter')Classification
# Tune the models, LightGBM: Classification lightgbm_tune = tune_model(lightgbm, fold=5) # Show the tuned hyperparameters, for example for LightGBM: Classification plot_model(lightgbm_tune, plot='parameter')PyCaret lets the users manually perform ensemble methods, like Bagging, Boosting, Stacking, or Blending. The below code performs each ensemble method.
Regression
# Bagging BR br_bagging = ensemble_model(br_tune, fold=5) # Boosting BR br_boost = ensemble_model(br_tune, method='Boosting', fold=5) # Stacking with Huber as the meta-model stack = stack_models(caret_models_5, meta_model=huber, fold=5) # Blending top models caret_blend = blend_models(estimator_list=[br_tune,huber_tune,omp_tune,ridge_tune,par_tune])Classification
# Bagging LightGBM lightgbm_bagging = ensemble_model(lightgbm_tune, fold=5) # Boosting LightGBM lightgbm_boost = ensemble_model(lightgbm_tune, method='Boosting', fold=5) # Stacking with ET as the meta-model stack = stack_models(caret_models_5, meta_model=et, fold=5) # Blending top models caret_blend = blend_models(estimator_list=[lightgbm_tune,rf,dt])Now, let’s choose blending models as the predictive models. The following code uses the blending models to predict the validation datasets.
Regression
# Predict the validation data caret_pred = predict_model(caret_blend, data = trainSet2.drop(columns=['SalePrice']))Classification
# Predict the validation data pred_caret = predict_model(caret_blend, data = trainSet2.drop(columns=['Target'])) AutoViMLI run AutoViML in the notebook by assigning many arguments. Just like PyCaret, AutoViML does not need splitting the features (X_train) and label (y_train). Users only need to split the training dataset (as trainSet1) and the validation dataset (as trainSet2). Users can specify other parameters, like scoring parameter, hyperparameter, feature reduction, boosting, binning, and so on.
Regression
!pip install autoviml !pip install shap from autoviml.Auto_ViML import Auto_ViML # Create the model viml, features, train_v, test_v = Auto_ViML(trainSet1, 'SalePrice', trainSet2.drop(columns=['SalePrice']), scoring_parameter='', hyper_param='RS', feature_reduction=True, Boosting_Flag=True, Binning_Flag=False,Add_Poly=0, Stacking_Flag=False, Imbalanced_Flag=True, verbose=1)Classification
!pip install autoviml !pip install shap from autoviml.Auto_ViML import Auto_ViML # Create the model viml, features, train_v, test_v = Auto_ViML(trainSet1, 'Target', trainSet2.drop(columns=['Target']), scoring_parameter='balanced_accuracy', hyper_param='RS', feature_reduction=True, Boosting_Flag=True, Binning_Flag=False,Add_Poly=0, Stacking_Flag=False, Imbalanced_Flag=True, verbose=1)After fitting the training data, we can examine what has been done from the output. For example, we can see that AutoViML was doing preprocessing by filling the missing data. Similar to MLJAR, AutoViML also gives the visualized report. For the regression task, it visualizes the scatter plot of true and predicted values using the XGBoost model. It also plots the prediction residual error in a histogram. From the two plots, we can observe that the model has relatively good accuracy and the residual error is a normal distribution.
The next report is ensemble method trials. We can find this report for both regression and classification tasks. The last graph displays the feature importance. We can see that the most important feature to predict house prices is exterior material quality, followed by overall material and finish quality, and size of the garage respectively.
Fig. 7 Scatter plot of true and predicted values, and residual error histogram. Fig. 8 Feature importances.
As for the classification task, it does not visualize the scatter plot, but the ROC curve for each of the four prediction classes, micro-average, and macro-average as well as the AUC value. We can see that all of the classes have AUC values above 0.98. The next chart reports the iso-f1 curves as the accuracy metric. It also later gives the classification report.
Fig. 9 ROC curves and iso-f1 curves
Average precision score, micro-averaged over all classes: 0.97 Macro F1 score, averaged over all classes: 0.83 ##################################################### precision recall f1-score support 0 0.84 0.97 0.90 963 1 0.51 0.59 0.54 258 2 0.18 0.11 0.14 191 3 0.00 0.00 0.00 118 accuracy 0.73 1530 macro avg 0.38 0.42 0.40 1530 weighted avg 0.64 0.73 0.68 1530 [[938 19 6 0] [106 151 1 0] [ 53 117 21 0] [ 16 12 90 0]]The following code prints out the result of AutoViML search. We can see that the best model is XGBRegressor and XGBClassifier for each task.
vimlRegression
XGBRegressor(base_score=0.5, booster='gbtree', colsample_bylevel=1, colsample_bynode=1, colsample_bytree=0.7, gamma=1, gpu_id=0, grow_policy='depthwise', importance_type='gain', interaction_constraints='', learning_rate=0.1, max_delta_step=0, max_depth=8, max_leaves=0, min_child_weight=1, missing=nan, monotone_constraints='()', n_estimators=100, n_jobs=-1, nthread=-1, num_parallel_tree=1, objective='reg:squarederror', predictor='cpu_predictor', random_state=1, reg_alpha=0.5, reg_lambda=0.5, scale_pos_weight=1, seed=1, subsample=0.7, tree_method='hist', ...)Classification
CalibratedClassifierCV(base_estimator=OneVsRestClassifier(estimator=XGBClassifier(base_score=None, booster='gbtree', colsample_bylevel=None, colsample_bynode=None, colsample_bytree=None, gamma=None, gpu_id=None, importance_type='gain', interaction_constraints=None, learning_rate=None, max_delta_step=None, max_depth=None, min_child_weight=None, missing=nan, monotone_constraints=None, n_estimators=200, n_jobs=-1, nthread=-1, num_parallel_tree=None, objective='binary:logistic', random_state=99, reg_alpha=None, reg_lambda=None, scale_pos_weight=None, subsample=None, tree_method=None, use_label_encoder=True, validate_parameters=None, verbosity=None), n_jobs=None), cv=5, method='isotonic') LightAutoMLNow, we have reached the 10th AutoML. LightAutoML is expected to be light as its name explains. Here, we set the task to be ‘reg’ for regression, ‘multiclass’ for multiclass classification, or ’binary’ for binary classification. We can also set the metrics and losses in the task. I set the timeout to be 3 minutes to let it find the best model. After simple .fit and .predict, we can already get the result.
Regression
!pip install openpyxl from lightautoml.automl.presets.tabular_presets import TabularAutoML from lightautoml.tasks import Task # Create the model light = TabularAutoML(task=Task(‘reg’,), timeout=60*3, cpu_limit=4) train_data = pd.concat([X_train, y_train], axis=1) # Fit the training data train_light = light.fit_predict(train_data, roles = {‘target’: ‘SalePrice’, ‘drop’:[]}) # Predict the validation data pred_light = light.predict(X_val)
Classification
!pip install openpyxl from lightautoml.automl.presets.tabular_presets import TabularAutoML from lightautoml.tasks import Task train_data = pd.concat([X_train, y_train], axis=1) # Create the model light = TabularAutoML(task=Task(‘multiclass’,), timeout=60*3, cpu_limit=4) # Fit the training data train_light = light.fit_predict(train_data, roles = {‘target’: ‘Target’}) # Predict the validation data pred_light = light.predict(X_val)
The results for the classification task are the predicted class and the probability of each of the classes. In other words, the result fulfills multiclass and multilabel classification expectations.
# Convert the prediction result into dataframe pred_light2 = pred_light.data pred_light2 = pd.DataFrame(pred_light2, columns=['4','2','3','1']) pred_light2 = pred_light2[['1','2','3','4']] pred_light2['Pred'] = pred_light2.idxmax(axis=1) pred_light2['Pred'] = pred_light2['Pred'].astype(int) pred_light2.head()1 2 3 4 Pred
0 0.00 0.01 0.00 0.99 4
1 0.00 0.00 0.00 1.00 4
2 0.00 0.04 0.00 0.96 4
3 0.00 0.01 0.98 0.01 3
4 0.02 0.38 0.34 0.27 2
ConclusionWe have discussed 10 AutoML packages now. Please notice that there are still more AutoML not yet discussed in this post. Not only in Python notebook but AutoML also can be found in cloud computing or software. Which AutoML is your favourite? Or, do you have other AutoML in mind?
Fig. 10 AutoML chúng tôi you still going to run 10 conventional Machine Learning or prefer the 10 AutoML packages?
About Author
References
Fig3- Image by Author
Fig4-Image by Author
Fig5-Image by Author
Fig6-Image by Author
Fig10- sklearn, topot, hyperopt, autokeras, mljar, autogluon, h2o, pycaret, autoviml
Connect with me here.
The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.
Related
Update the detailed information about Google’s Automated Image Captioning & The Key To Artificial “Vision” on the Hatcungthantuong.com website. We hope the article's content will meet your needs, and we will regularly update the information to provide you with the fastest and most accurate information. Have a great day!