eduzhai > Applied Sciences > Computer Science >

Estimating Network Effects Using Naturally Occurring Peer Notification Queue Counterfactuals

  • KanKan
  • (0) Download
  • 20210425
  • Save

... pages left unread,continue reading

Document pages: 18 pages

Abstract: Randomized experiments, or A B tests are used to estimate the causal impactof a feature on the behavior of users by creating two parallel universes inwhich members are simultaneously assigned to treatment and control. However, insocial network settings, members interact, such that the impact of a feature isnot always contained within the treatment group. Researchers have developed anumber of experimental designs to estimate network effects in social settings.Alternatively, naturally occurring exogenous variation, or naturalexperiments, allow researchers to recover causal estimates of peer effectsfrom observational data in the absence of experimental manipulation. Naturalexperiments trade off the engineering costs and some of the ethical concernsassociated with network randomization with the search costs of findingsituations with natural exogenous variation. To mitigate the search costsassociated with discovering natural counterfactuals, we identify a commonengineering requirement used to scale massive online systems, in which naturalexogenous variation is likely to exist: notification queueing. We identify twonatural experiments on the LinkedIn platform based on the order of notificationqueues to estimate the causal impact of a received message on the engagement ofa recipient. We show that receiving a message from another member significantlyincreases a member s engagement, but that some popular observationalspecifications, such as fixed-effects estimators, overestimate this effect byas much as 2.7x. We then apply the estimated network effect coefficients to alarge body of past experiments to quantify the extent to which it changes ourinterpretation of experimental results. The study points to the benefits ofusing messaging queues to discover naturally occurring counterfactuals for theestimation of causal effects without experimenter intervention.

Please select stars to rate!


0 comments Sign in to leave a comment.

    Data loading, please wait...