Performance Quality Models (or PQM in short) are a collection of AI models with a bold goal to predict various performance proxies (e.g. page load time (PLT) class) of any web / mobile page. Just like netinfo’s effective type attribute, they predict, say the PLT class of any HTTP request as good / poor. Their difference is explained in the FAQ section below. As PQMs use both device and network characteristics while predicting the PLT (or any proxy) class, it can be used to build impactful application features that are otherwise tricky to implement. Some examples are:
- Dynamically adjust the quality of images based on the current PLT class. Read this blog to learn more about our experience and results.
- Disable autoplay of videos on feed or elsewhere for slow devices
- Reduce the size of payload for poorer networks [1]
These model files are binaries that can be imported into your Python or node.js backends. Theoretically, this model file can also be imported in your client side JavaScript / Android / iOS code just like any other assets. We do not show that approach here, but please reach out to us if that suits your use case better.
This is our first attempt to open source a ML model trained on LinkedIn’s performance data of millions of page views around the world. So, please share your experiences using these models in the discussion forum. If you have ideas around how to distribute these models better or how to improve their accuracy, don't hesistate to reach out!
Go to the corresponding folders of each model to see how to use them, how they were built and other useful information. We will continue to update this list as we build and test new models thoroughly.
If you use server side rendering or static file server techniques to serve your mobile web traffic, you can use these models.
- How LinkedIn Personalized Performance for Millions of Members using TensorFlow.js published in March 2022 by Google talks about a clever way to deploy these models into production without any complex AI stack
- Understanding network quality: The rise of customized content delivery
- Personalizing Performance: Adapting Application in real time to member environments
- Discussions with W3C on the future of Netinfo
- How LinkedIn uses AI to optimize performance for every member at Facebook Performance Summit in 2019 - slides | recording
- AutoML to find the best PQMs at Ray Summit in 2020 - slides | recording
1. Aren't on-device realtime measurements calculated by the NetInfo API going to be more accurate than a prediction generated by a model that was trained with historical data? And if so, what are the advantages of using a predictive model instead? Also, it looks like there are some other options besides NetInfo like https://github.com/bluesmoon/boomerang/ or https://github.com/sbstjn/latenz
NetInfo or similar only consider network aspects, whereas PQMs include network, server, client and CDNs' performance as a whole.
NetInfo has privacy concerns and not a standard web API yet. It is not available on Safari and many other browsers, which can be a huge drawback in countries where iOS is a big market.
Having said that, we plan to include real time network measurements as inputs to the model in the future to further improve its accuracy.
2. I’m new to ML and not sure if Linkedin’s PQMs would work for my company’s problems. How can I quickly verify this?
PQMs, though trained on LinkedIn data, capture users’ devices and network conditions all around the world. As we did not use any crucial and private information to LinkedIn to either train or test them, we believe the models captured the general network and device characteristics of users. These characteristics would not change much whether a user visits linkedin.com or medium.com on the same device and from the same place. Just like any 3rd party software, give it a try on your own data, check its performance and report any problems in the issue tab. It is very simple to try these models. Please see example scripts of each model for more information.
If you wish to customize the model, techniques like transfer learning can be used. With transfer learning, you can further fine tune and retrain the model on your own performance datasets. As the model already has a general sense of performance, fine tuning wouldn’t need a lot of data or training infrastructure.
Unlike general programming, ML models tend to lose their predictive power with time. As we also use them inside LinkedIn, we constantly monitor their accuracy. When we see it fall, we will continue to release newer versions of the models.
Feel free to file an issue if you want more advice or help on any of the above steps to try. We are happy to collaborate :)
3. How does the accuracy of a predictive model compare to on-device real-time measurements?
NetInfo, an experimental API available on Chromium based browsers uses on-device measurements to estimate the future throughput and RTT of network requests. Page Speed Predictor on the other hand estimates the overall page load performance which includes network, server and client latencies. So there is no apples to apples comparison and depends on the use case of how the prediction is used.
4. Is this model currently being used at LinkedIn? If so, what is it being used for? If not, why not?
Lite, the global mobile web offering of LinkedIn, uses PQM to decide the resolution of images on feed today in realtime for every request. We shared our experiences and early results at multiple venues like Facebook's Performance Summit in 2019 and Ray Summit in 2020.
5. The model in the git repository was last updated in Jan 2022. When will it next be updated?
We actively monitor the accuracy of the model on a daily basis and will continue to publish new versions whenever we see a drop in those metrics. We can encourage you to open an issue if you see a drop in model's performance.
Tensorflow
https://github.com/tensorflow/tensorflow
Copyright 2019 The TensorFlow Authors
License: Apache License 2.0