Big-scale add overall performance checks with the PageSpeed ​​Insights and Python APIs

Google offers the APISpeed ​​Insights to assist seo specialists and builders by using mixing actual statistics with simulation information and providing load time records associated with web sites.

The distinction among Google PageSpeed ​​Insights (PSI) and Lighthouse is that PSI consists of both actual and lab information, whilst Lighthouse performs a web page loading simulation with the aid of converting the connection and consumer agent of the tool.

some other distinction is that PSI does not provide any data related to web accessibility, seo or revolutionary internet applications (PWA), whilst Lighthouse offers all the above.

hence, while we use the PageSpeed ​​Insights API to test the overall performance of URL uploads on a huge scale, we will not have any accessibility statistics.

but, PSI provides more records associated with page velocity, together with “DOM length,” “inner most infant DOM detail,” “total jobs,” and “DOM content loaded time.”

some other benefit of the PageSpeed ​​Insights API is that it offers “located measurements” and “actual measurements” distinct names.

in this manual you will study:

• the way to create a Python script on the production degree.
• the way to use APIs with Python.
• a way to compile statistics frames from API responses.
• How to analyze API responses.
• a way to parse URLs and manner responses to URL requests.
• how to store API responses with the ideal shape.

An example of a web page speed ​​Insights API call exit with Python is below.

Libraries for the usage of the PageSpeed ​​Insights API with Python

the specified libraries to use the PSI API with Python are under.

• advertising gear retrieve test URLs from Sitemap.
• Pande is to collect a facts body and equalize the output of the JSON API.
• The necessities are to require a specific API endpoint.
• The JSON must take the API response and enter it in a particularly related dictionary object.
• Datetime is to exchange the name of a particular output file with the date of the instant.
• URLlib is to parse the URL of the check difficulty’s internet site.

how to use PSI API with Python?

to apply the PSI API with Python, comply with the steps below.

• achieve the PageSpeed ​​Insights API key.
• Import the desired libraries.
• damage down the URL for the check concern’s website.
• Take the Date of the instant as the report name.
• Get the URLs listed from the Sitemap.
• select the metrics you want from the PSI API.
• Create a For loop to just accept the API response for all URLs.
• build a information body with selected PSI API metrics.
• Print the effects in XLSX format.

1. Acquire the PageSpeed ​​Insights API key

Use PageSpeed ​​Insights API Documentation for acquiring the API key.

click the “Get Key” button below.

pick out the venture you created inside the Google Developer Console.

enable the PageSpeed ​​Insights API for a particular challenge.

you will want to use a special API key in your API necessities.

2. Import the desired libraries

Use the strains below to import simple libraries.

    import advertools as adv
    import pandas as pd
    import requests
    import json
    from datetime import datetime&#thirteen;
    from urllib.Parse import urlparse

three. Break down the URL for the difficulty’s internet site

Use the code shape below to interrupt down the URL of the site in query.

  area = urlparse(sitemap_url)
  area = area.Netloc.Cut up(".")[1]

The “domain” variable is a parsed version of the Sitemap URL.

“Netloc” represents the area section of a selected URL. When we divide it with the aid of “.” occupies the “center part” that represents the domain call.

here is “0” for “www”, “1” for “area call” and “2” is for “area extension” if divided by using “”.

four. Take the date of the instant because the report name

To take the date of a particular feature name time, use the “datetime.Now” technique.

Datetime.Now offers a selected time of a selected time. Use “strftime” with values ​​”% Y”, “% m” and “% d”. “% Y” is for year. “% M” and “% d” are numeric values ​​for a selected month and day.

 date = datetime.Now().Strftime("%Y_p.Cm_p.Cd")

5. Take the URLs to the listing from the Sitemap

To switch URLs to a listing form from a Sitemap document, use the code block under.

   sitemap = adv.Sitemap_to_df(sitemap_url)
   sitemap_urls = sitemap["loc"].To_list()

in case you read Python Sitemap health reviewyou can find out greater statistics approximately Sitemaps.

6. Pick out the metrics you want from the PSI API

To select the JSON homes of the PSI API reaction, you need to most effective see the JSON file.

it is very crucial for studying, parsing and leveling JSON items.

that is even associated with Semantic seoway to the concept of “directed graph” and established “JSON-LD” records.

In this text, we are able to no longer recognition on examining precise JSON response hierarchies of the PSI API.

you could see the measurements I selected to acquire from the PSI API. It is richer than the primary default output of the PSI API, which best offers middle net Vitals or pace ​​Index interplay metrics for the next color, time to first byte, and primary content colour.

Of course, it also gives “pointers” with the word “avoid chaining essential necessities,” but there’s no want to insert a sentence into the facts body.

within the future, these templates, or even every individual chain occasion, their KB and MS values, can be taken in a unmarried column called “psi_suggestions”.

For starters, you may test out the metrics I’ve selected, and a vast amount of them may be first for you.

PSI API Metrics, the primary section is under.

    fid = []&#thirteen;
    lcp = []&#thirteen;
    cls_ = []
    url = []
    fcp = []&#thirteen;
    performance_score = []
    total_tasks = []
    total_tasks_time = []&#thirteen;
    long_tasks = []&#thirteen;
    dom_size = []
    maximum_dom_depth = []
    maximum_child_element = []&#thirteen;
    observed_fcp  = []&#thirteen;
    observed_fid = []
    observed_lcp = []&#thirteen;
    observed_cls = []&#thirteen;
    observed_fp = []&#thirteen;
    observed_fmp = []
    observed_dom_content_loaded = []
    observed_speed_index = []
    observed_total_blocking_time = []&#thirteen;
    observed_first_visual_change = []
    observed_last_visual_change = []
    observed_tti = []&#thirteen;
    observed_max_potential_fid = []

This section consists of all observed and simulated fundamental page velocity measurements, along side some non-middle ones, including “DOM content loaded” or “the primary realistic colour. ”

the second segment of PSI Metrics makes a speciality of viable byte and time savings due to unused quantity of code.

    render_blocking_resources_ms_save = []
    unused_javascript_ms_save = []
    unused_javascript_byte_save = []
    unused_css_rules_ms_save = []
    unused_css_rules_bytes_save = []

The third a part of PSI measurements focuses on server response time, the benefits of the use of a response picture or not, the usage of harms.

    possible_server_response_time_saving = []
    possible_responsive_image_ms_save = []

notice: the overall overall performance rating is based totally on “performance_score”.

7. Create a For loop to just accept the API response for all URLs

The for loop is to take all the URLs from the sitemap report and use the PSI API for all of them one at a time. The for loop for the PSI API automation has numerous sections.

the primary part of the PSI API for loop begins with preventing reproduction URLs.

you may see a URL that appears more than one times in Sitemaps. This segment prevents this.

for i in sitemap_urls[:9]:&#thirteen;
         # save you the duplicate "/" trailing diminish URL requests to override the statistics.&#thirteen;
         if i.Endswith("/"):&#thirteen;
               r = requests.Get(f"https://www.Googleapis.Com/pagespeedonline/v5/runPagespeed?Url=i&method=cellular&locale=en&key=api_key")
         else:
               r = requests.Get(f"https://www.Googleapis.Com/pagespeedonline/v5/runPagespeed?Url=i/&approach=cell&locale=en&key=api_key")

do not forget to test “api_key” on the endpoint for the PageSpeed ​​Insights API.

test the repute code. Sitemaps may comprise non-2 hundred popularity code URLs; those want to be wiped clean.

         if r.Status_code == 2 hundred:
               #print(r.Json())&#thirteen;
               data_ = json.Masses(r.Text)&#thirteen;
               url.Append(i)

the next segment adds special metrics to the special dictionary we created before “_data”.

               fcp.Append(data_["loadingExperience"]["metrics"]["FIRST_CONTENTFUL_PAINT_MS"]["percentile"])
               fid.Append(data_["loadingExperience"]["metrics"]["FIRST_INPUT_DELAY_MS"]["percentile"])&#thirteen;
               lcp.Append(data_["loadingExperience"]["metrics"]["LARGEST_CONTENTFUL_PAINT_MS"]["percentile"])&#thirteen;
               cls_.Append(data_["loadingExperience"]["metrics"]["CUMULATIVE_LAYOUT_SHIFT_SCORE"]["percentile"])&#thirteen;
               performance_score.Append(data_["lighthouseResult"]["categories"]["performance"]["score"] * a hundred)

the next phase makes a speciality of the number of “shared duties” and the size of the DOM.

               total_tasks.Append(data_["lighthouseResult"]["audits"]["diagnostics"]["details"]["items"][0]["numTasks"])
               total_tasks_time.Append(data_["lighthouseResult"]["audits"]["diagnostics"]["details"]["items"][0]["totalTaskTime"])&#thirteen;
               long_tasks.Append(data_["lighthouseResult"]["audits"]["diagnostics"]["details"]["items"][0]["numTasksOver50ms"])
               dom_size.Append(data_["lighthouseResult"]["audits"]["dom-size"]["details"]["items"][0]["value"])

the following segment offers with “Dom Globina” and “The inner most detail of DOM”.

               maximum_dom_depth.Append(data_["lighthouseResult"]["audits"]["dom-size"]["details"]["items"][1]["value"])&#thirteen;
               maximum_child_element.Append(data_["lighthouseResult"]["audits"]["dom-size"]["details"]["items"][2]["value"])

the subsequent section discusses the unique observed test consequences for the duration of our page speed ​​Insights API.

               observed_dom_content_loaded.Append(data_["lighthouseResult"]["audits"]["metrics"]["details"]["items"][0]["observedDomContentLoaded"])&#thirteen;
               observed_fid.Append(data_["lighthouseResult"]["audits"]["metrics"]["details"]["items"][0]["observedDomContentLoaded"])
               observed_lcp.Append(data_["lighthouseResult"]["audits"]["metrics"]["details"]["items"][0]["largestContentfulPaint"])
               observed_fcp.Append(data_["lighthouseResult"]["audits"]["metrics"]["details"]["items"][0]["firstContentfulPaint"])&#thirteen;
               observed_cls.Append(data_["lighthouseResult"]["audits"]["metrics"]["details"]["items"][0]["totalCumulativeLayoutShift"])
               observed_speed_index.Append(data_["lighthouseResult"]["audits"]["metrics"]["details"]["items"][0]["observedSpeedIndex"])
               observed_total_blocking_time.Append(data_["lighthouseResult"]["audits"]["metrics"]["details"]["items"][0]["totalBlockingTime"])
               observed_fp.Append(data_["lighthouseResult"]["audits"]["metrics"]["details"]["items"][0]["observedFirstPaint"])
               observed_fmp.Append(data_["lighthouseResult"]["audits"]["metrics"]["details"]["items"][0]["firstMeaningfulPaint"])&#thirteen;
               observed_first_visual_change.Append(data_["lighthouseResult"]["audits"]["metrics"]["details"]["items"][0]["observedFirstVisualChange"])&#thirteen;
               observed_last_visual_change.Append(data_["lighthouseResult"]["audits"]["metrics"]["details"]["items"][0]["observedLastVisualChange"])
               observed_tti.Append(data_["lighthouseResult"]["audits"]["metrics"]["details"]["items"][0]["interactive"])
               observed_max_potential_fid.Append(data_["lighthouseResult"]["audits"]["metrics"]["details"]["items"][0]["maxPotentialFID"])

the next section incorporates the amount of unused code and misplaced bytes in milliseconds together with render blocking resources.

               render_blocking_resources_ms_save.Append(data_["lighthouseResult"]["audits"]["render-blocking-resources"]["details"]["overallSavingsMs"])&#thirteen;
               unused_javascript_ms_save.Append(data_["lighthouseResult"]["audits"]["unused-javascript"]["details"]["overallSavingsMs"])
               unused_javascript_byte_save.Append(data_["lighthouseResult"]["audits"]["unused-javascript"]["details"]["overallSavingsBytes"])
               unused_css_rules_ms_save.Append(data_["lighthouseResult"]["audits"]["unused-css-rules"]["details"]["overallSavingsMs"])&#thirteen;
               unused_css_rules_bytes_save.Append(data_["lighthouseResult"]["audits"]["unused-css-rules"]["details"]["overallSavingsBytes"])

the following section is to provide the blessings of the reaction image and server reaction time.

               possible_server_response_time_saving.Append(data_["lighthouseResult"]["audits"]["server-response-time"]["details"]["overallSavingsMs"])      
               possible_responsive_image_ms_save.Append(data_["lighthouseResult"]["audits"]["uses-responsive-images"]["details"]["overallSavingsMs"])

the next phase is that the characteristic keeps to work within the occasion of an error.

         else:
           keep

example of the usage of the web page velocity ​​Insights API with Python for mass trying out

to apply precise code blocks, insert them into the Python characteristic.

Run the script and in the columns below you may get 29 measurements related to page velocity.

end

The PageSpeed ​​Insights API offers different sorts web page load performance measurements.

It indicates how Google engineers perceive the concept of page load overall performance and possibly use these metrics as a point of view for ranking, UX, and knowledge excellent.

the usage of Python for mass page velocity checks offers you a image of the complete web site to assist analyze capacity person experience, move slowly overall performance, conversion fee, and ranking enhancements.

more assets:

---

decided on image: Dundanim / Shutterstock