SIGCOMM13: Developing a Predictive Model of Quality of Experience for Internet Video

The paper was presented by Athula Balachandran.

Other co-authors are: Vyas Sekar, Aditya Akella, Srinivasan Seshan, Ion Stoica and Hui Zhang.

The metrics to measure Internet video QoE have shifted from traditional method to new method. This paper is about developing a predictive model of Internet video QoE. The model meets two requirements: (1) it has to be tied in to observable user engagement; (2) it should be actionable to guide practical system design decisions.

Commonly used quality metrics include join time, buffering ratio, rate of buffering, rate of switching, average bitrate, etc. However, which metrics should we use for QoE are unknown. This work develops a unified and quantitative QoE model to solve the problem.

Challenge:

• complex engagement-to-metric relationships and complex metric interdependencies
• identify confounding factors
• incorporate confounding factors

Approaches:

• Cast complex relationships as a machine learning problem.
• Design different test to examine potential factors and then identify key confounding factors.
• Two methods are proposed to refine the model to incorporate confounding factors: add confounding factors as a feature and split the data by confounding factors. The speaker shows that split is better than feature, allowing the model to achieve 70% accuracy.

Evaluation:

The speaker shows that there is 100% improvement of average engagement comparing with baseline and 20% improvement comparing with other strategies.

Q: Is there any correlation between some confounding factors?

A: Some correlations are seen because of user behaviors.

Q: how does this approach capture real user effects in the wild?

A: The data used in the analysis is from real user data.

Q: Why not use other machine learning models?

A: Could use.

More details can be found:

http://conferences.sigcomm.org/sigcomm/2013/papers/sigcomm/p339.pdf