He mainly created a hierarchy of disagreements as explained in the diagram below

You come across many such examples on a day to day basis. For example here is a recent one I saw on threads, and disagreements(Some real some made up)

Point: 

Someone posted on the lines of : 
while meta’s new social media threads is good , its not yet able to replace twitter.Twitter is still where all political conversations happen. Twitter won

Disagreements Hierarchy levels

DH0: Cry harder you liar  → Adds no value

DH1: Very telling that your source bought a golden check on twitter and has to now justify the investment → Attacks the source rather than the story

DH2: Weird that you are celebrating this  → Attacks imagined tone

DH3: Not true. I find all my news here. I login 27 times a day  → Anecdotal  

DH4: Twitter has lost most of the advertisers and most of the top news accounts have moved out. Threads on the other hand has 100M DAU.
→ Twitter might have lost advertisers and the news accounts , but that doesnt actually mean that political conversation is not happening.Threads doing well is also not exactly relevant to the argument. Its a parallel conversation

DH5: Threads has much fewer users than twitter at the moment so surely the volume would be lower. But if you were to look at this report . Threads generated XX million impressions and engagement on political content and on a per user basis this is far higher than what we see on twitter  . If we extrapolate this based on current user growth , threads is likely to overtake twitter sometime end of this year
→ A good argument that talks about per user engagement and how it is much healthier than twitter. While it is still not better in terms of overall numbers, it’s very likely getting there. This argument does have the power to change someones mind

DH6: Threads has already stated its intentions to not be a platform for mainly political discourse at this and this place. It actively suppresses any political content, discouraging anyone from posting. If threads intention was always to not complete in this space, expecting it to “beat” twitter seems unrealistic
→ Refutes the whole comparison itself. You win or lose when you are playing and this disagreement demonstrates that threads does not even intent to beat twitter

The main thesis was that while these disagreement hierarchies are not necessarily for picking the winners or losers, they are a great filter for which disagreements to engage with

Over the years, as the Internet has become more and more crowded this problem has compounded by several orders of magnitude .

When can you move the conversation

I believe it would be fair to say that most of us would want to disagree at the very highest level ie “refuting the central point”

This is where I have seen well meaning some people try and move the conversation up the hierarchy. I also have tried this on multiple occasions (Though I agree I also sometimes enter disagreements at much lower levels)

For example if somebody were to simply make a counter argument which many a times is not the argument that you were making, but rather some kind of an adjacent argument, you would try to bring back .
For eg refer to the “counterargument stage earlier”

Your strategy here could be

• restating your point: While you are right on advertisers, twitter is still the place for political content .

• making it narrow and making it clearer. : Ignore advertisers , or news accounts. Let’s only talk about actual users engaging in political conversations . Twitter is still the place

• Ignoring the distraction and not taking the bait to engage in the parallel adjacent argument.

You assume good intent and try to clarify

You do this in hopes that it moves from counter argument to an actual refutation or refuting the central point. I have tried this as an experiment in multiple disagreements I have had with people online, but I have found that this mostly doesn’t really work. 

The chances of a disagreement moving towards higher parts of the pyramid are much less than moving lower. 

Any conversation that goes beyond two or three exchanges and does not move up will almost certainly fall down rapidly, eventually till name calling. Sometimes it will skip many  steps in between. 

The first few exchanges often determine whether the conversation will climb the hierarchy or slide down it

There also seems to be a point of no return in many online disagreements. Once a conversation crosses this threshold, it’s pretty tough to get it back on track

That point potentially is somewhere in the counterargument stage and within that counterargument itself. Perhaps Paulg should update his hierarchy to split the counter argument stage into two parts

No common ground counterargument: Irrespective of how well the counterargument is put, if it is not addressing exactly what the author initially stated, its no-common ground argument
Eg:Twitter has lost most of the advertisers and most of the top news accounts have moved out

Common ground counterargument : Irrespective of what the counterargument  is, as long as it is talking about the same argument it is common ground. The counterargument need not be convincing, or well worded . Its simply needs to address the main point
Eg: Conversation is mostly driven by news accounts. Most top news accounts have moved out and Wall street journal reports that CTRs through twitter have halved

Anything below and including the no common ground counter argument is likely to devolve into name calling with an occasional cancelling attempt . Let’s call it the no hope zone, Or the cancel zone

While all arguments can be up-leveled, the probability below and including no common ground counterargument have much less probability of moving up and much higher to move down. It will take enormous patience and effort to up level that conversation, and it might not be ROI positive

• If the first few arguments by the person you are disagreeing with are not in the no hope zone, then you can in good faith try and up level it. See where it goes, and if it changes someones mind

• If its in the no hope zone, either ignore or start collecting your best fun insults because that’s where it is going. I love :“ I do not have the patience or the crayons to explain this to you”

 It is still Paul Graham’s hierarchy of disagreement but with a filter on top .

The internet is a vast and noisy place. It's easy to get sucked into unproductive arguments. But by being more mindful of the quality of our own arguments and more selective about the disagreements we engage in, we can raise the level of discourse.

Happy disagreeing

PS: I am on threads

And now the poetry break:

This blog is called product and poetry 🙂 And I plan to share some poetry or haikus or ghazals or shers with every newsletter. If it is not in english, I will also give you the meaning

This post is mostly relevant for larger big tech companies with products that spawn years / decades and have a rich legacy. Eg: Think Apple App store, think Amazon cart

Whenever you join one of these larger companies you end up owning a piece of product that has been owned by many PMs in the past and has undergone many iterations, decisions, and designs that have either not been documented or were documented but are kind of lost.

One issue that happens with a lot of New PMs is that they get overwhelmed and start “defending” themselves and “deflecting” the blame to their predecessors whenever asked about certain specific decisions. This would happen typically when another team wants to do some changes and wants your opinion on your part the product, or when a customer escalates and you are looped in .

Some statements you would hear:
-I have no clue why this was done, I would not have done that.
-This is ridiculous and makes no sense. Lets remove it
-Whats my take on this? I have none, this was before my time and i dont care if you want to remove it

The problem with this approach is that , for good or for worse , you now OWN THAT DECISION .
But how do you OWN a decision you didnt make , and that my friend is a great question . Here is the strategy

1. You did not make that decision and thats ok: This is something that is clear to everyone. You do not need to know the answer for everything. You also need to internalize this, if someone thinks this design makes no sense its not a comment on you, its just a comment. Your job is what now

2. Figure out : Take a pause and figure out WHY that decision was taken and what are the repercussions. Seems pretty commonsensicle but a lot of new PMs attempt to jump straight to having a POV.
   Some things to check
   
   Legal / policy concerns?: Maybe there was a specific legal ask at that point in time? Maybe its still there. A lot of decisions are sometimes reactionary to the guidelines by various government agencies
   
   Eng concerns : Maybe the systems at that point were not sophisticated enough? Eg: It makes sense to show your scheduled uber rides on the home page, but when it was built it was not big enough use case to make that additional API call to a different system?
   
   Just the best idea at that time : Sometimes the team at that time did the best that was possible. It was a judgement call
   
   Specific market conditions? You would be surprised at how many features get built to address specific market needs which end up being temporary but the feature never get deprecated.
   Maybe your predecessor built this in reaction to a very big partner and that specific design eventually became the generic standard?
   
   USAGE USAGE USAGE : How many users really use it? It has surprised me multiple times at how many tiny features here and there are not being actively looked at. They are being monitored ie if it goes down some eng team will know and fix it , but no ones really checking much.
   
   Tangentially its also possible that the internal analytics systems have been updated and that specific feature is not even being measured . Large companies also have strict data retention policies which mean a lot of historical data is just not there . I have built new features basing my hypothesis on as little as 30 days of data , sometimes no data at all. Gut based calls

3. Your POV
   Now the FUN part . Have answers to these
   – Do you agree with what you saw?
   – What happens if you remove it
   – Has something else been built that can replace this older functionality ( It very frequently is)
   -If it was to address a need, is that still true?
   – What next: I have deprecated features, and sometimes doubled down because we found a gem due to these explorations. In the end you need to finally have a POV.

If you want to be seen as the owner of that product area, it comes with taking ownership of all the decisions of the past, good, bad and the stupid

I used to love twitter, maybe too much.
Ps: Now I am on Threads

One of the reasons was not just because it seemed like a great place for content consumption but also a great place for content publishing. You could express yourself in short tweets or tweet threads, you could get instant feedback, you could “produce” more content and it was a pure win win

I used to think these were great things because

• Low bar of entry. Writing a blog post is hard. It takes time and effort. Tweets are “full blown” ideas expreseed in 240 characters. Much more easier to produce. I dont have to wait to spend time to share my thoughts

• I thought the ability to boil down complex topics in 240 characters was great as it forced me to focus on the bare essense of the topic without the fluff. I was kind of proud of mine

• and you could always thread tweets for slightly longer topics

• It was also easier to consume because when I read someone elses tweets / tweet threads it felt like they were talking about the bare essence of their idea without wasting my time

• Many of my tweet threads started becoming Blog posts, which was pretty good way of getting out of the writing rut

But as years went by I started to see some changes

• I started THINKING in tweets: Its one thing to be able to express your ideas in tweets, and another to start “Thinking” in tweet format. Its hard to explain but go with me here a bit

• I stopped self education IF I had enough to tweet : Since there is no nuance in tweets I had a fake sense of understanding of a topic if I had enough information to make a tweet. Even if I did not tweet about it, in my mind it started to feel “good enough” .
  I cannot possibly have great understanding of every new tech out there , but I started realising that I was stopping my education much earlier, in part because I knew I could “tweet” about it if needed.
  Even though it was not the aim, “can I tweet if needed” become a barometer of “if i understood” the topic . This was a LOW bar. For eg when I look back, I am horrified how little I knew about a topic I tweeted on but I can pretty much stand behind all my long forms

• I was consuming more, learning less. Reading through twitter threads of learned people started giving me this false sense of understanding. Do not get me wrong, some were great but many were just meant to be “tweets”

• I forgot tweets are much more temporary, there is little to look back on, little for people to “know you by” , little to refer after years. Blogs and long form are kind of more permanent, so If I do want to be known for some of my work, twitter is only the entry gate not the show

• My blog posts that were inspired by tweets started ending up just being slight expansions of the tweets themselves. They were not writing prompts but just disorganized thoughts put together in one giant pile of garbage blog . It was not useful.

So how do I plan to change?

• I want to retrain my brain from thinking in tweets / short form and start thinking long form.

• Ideas are messy and tweets are kind of structured. There is a reason blogs are long, and the core idea needs support, and that telling the “crux” directly doesn’t help. You need to go through the full journey to finally have some understanding . I want to embrace that messiness

• Tweets From blogs and not the other way round. I am still for threads / tweets as a great form of communication but it has to start as a long form and then become a tweet , not vice versa.

As some of you already know Twitter was perhaps my most favorite social media website. I loved using Twitter, and was, and still am very active (Though reducing)

I was excited when musk said that he wanted to buy the platform, but looking at all his shenanigans from trying to walk out of the deal to whatever he’s doing right now, makes me pretty pessimistic about the future of this platform..

Every time I think about writing about Twitter and my point of view on its latest and greatest features, they take a complete U-turn which makes the previous thing completely irrelevant.

Perhaps this is the reason I have not written about Twitter for a while, but I want to talk about Twitter blue and the credibility crisis it is facing

As some of you already know, Twitter introduce something called Twitter blue . When you subscribe to Twitter blue, apart from various features such as edit tweets, the main two things that really differentiate it from a normal users experience is the blue checkmark and an artificial boost to all your tweets.

Now at the surface, both of these things do not seem too wrong but there are some downside to this let’s deep dive

Artificial Boost

Not only tweets from blue are given an artificial boost in the timeline of the users, recently it was announced that only blue users might show up in users for you tab.

there are a few major implications of this.

Tweets as Ads :

Technically tweets from all blue users now are kind of ads. No I’m not against ads, ads are pretty cool, but they do have a very different connotation than tweets.
Some users like them, but many users may consider them low trust and start ignoring them. People already have something called ad blindness, I suspect that might happen with tweets from twitter blue

Food for thought : There are only 200-300K blue subscribers , and if the For you tab of 200M users is populated only with Blue subscriber content, you could potentially buy a few blue accounts and swamp their timelines. I argue that for an advertiser today it might be cheaper to simply buy a few blue accounts than actually running ads. This is a short term arbitrage which will close soon

Low trust for Twitter Blue tweets:

if you see a tweet come to your timeline organically, I bet you might still think it’s relevant or popular, and may be slightly more attention to it. On the contrary, if you see a tweet from a Twitter blue subscriber now show up on your timeline your first gut reaction might be that this person paid for this, while you still see that tweet, it’s likely that you would give it much less attention. I believe the hope is that this reduction in trust is overcome by the increased impressions of the tweet

The blue checkmark Status

The death of status:

let’s be very clear, the most likely reason Twitter created a verified program or a Twitter blue program was to prevent impersonation.
Twitter were sued by many celebrities for impersonation. Trying to mix the blue with the verified in my opinion was already a bad idea, and a lot of bad things have already happened. I am not going to go into those, but let’s just look at what Twitter blue stands for now.

Earlier Twitter blue stood for some kind of notability. A lot of people in my opinion bought blue thinking that they can also pretend to have that notability.

The problem that happened now is, Twitter and it’s in finite wisdom, and also because possibly musk had promised to, removed the blue check marks of all legacy verified people as well.

This leaves us with only people who want to buy blue for the specific feature set of blue, main one being boost and “status”.
This attracts a specific kind of audience, which may or may not contain the older legacy verified.

To make things worse, because of all the shenanigans that were done on Twitter in the past, buying Twitter blue in many circles started being looked at a negative status symbol.
Some people disliked it because of how musk treated other people , and some labeled everyone who bought as people who have something to sell.
To give you an sense of latter : think about your favorite or most hated influencer who buys ads to make sure that you see that contact.

I don’t fall in either camp but I too do not give any special credence to tweets by blue anymore. If anything, when i see a tweet by blue subs who I do not follow, my gut reaction is, paid ad.

After removing legacy, the only people that were left were people who bought and the only status left is : “This person paid “

The funny thing about status symbols is that you need people with actual status to use them.

And let’s be honest with ourselves, LeBron has far bigger status than pooja12345.

Twitter is slowly realising that the value to blue came FROM the people who had it, and not as much the other way round.

Blue was the perk that came with Status, not the other way around

Lebron did not need blue, but him having blue gave it credibility and made it aspirational
The status associated with blue was dead the moment legacy blue were removed

As I type this article now I have come to realize that Twitter is now randomly giving celebrities and largest accounts Twitter blue even though they explicitly said they don’t want it. This is perhaps to somehow restore some “status” to the blue checkmark temporarily so that more people buy.

This is not only ridiculous, but unethical. This is misleading advertisement , and potentially illegal

Think about it, how would you rate a startup that decides to put your picture on their profile as their paid user even though you never paid for it what about a start up that claims tesla Twitter and SpaceX, by their software, even when they never did. This is absolutely false advertising, and potentially lead to some interesting lawsuits.

Did you know?
Elon musk is a paid subscriber to my newsletter and so is every employee of twitter, spacex, and Tesla. You on the other hand can get it for free by using this link

The above statement is 100% false. Its a lie, it’s just a fabrication to drive home a point, dont sue me.I don’t even have paid subscription, but you should still subscribe

Should I be allowed to say the first statement without the second?

Subscribe now

Worst is that they are giving these ticks even to dead celebrity accounts. I don’t understand who greenflagged that

Things are so bad in some circles that celebrities are sharing tricks to to GET RID of the blue check mark.

But status loss has a catch

Status loss is slow and takes time to reach the masses. For Eg: Top quora voice was a status once, forbes 30under30 is slowly going away, Tedx speaker is basically meh . In the interim a scammer could easily fool people with a “bought” status. I do expect twitter to become more scammy and spammy because of this feature.

The only hope for blue possibly is if enough notable people subscribe, before it gets road rolled by spam and scam .

Will I buy twitter blue?
Well this question to me currently is similar to “will I buy ads”.
Sure if i have a need for it.

Eg If I finally write my book on product management, maybe.

But, one thing I will be well aware of is that my tweets now are kind of ads and that I have no real “status” because of blue.

A lot of my engagement would no longer be because of my banger tweets, but rather my 8$ subscription.

Do I judge you for taking blue
Well no, but
Assume my For You tab has two tweets from account I do not follow
1) From someone without blue
2) Your tweet with blue

My gut would automatically tag yours as possibly lower quality because your tweets did not fight millions of other tweets to be shown here, but rather are there because you paid for me to see it. But I will likely see much more of you. I think that is a fair tradeoff .

A good analogy is , assume you are a harvard student and harvard students are pretty smart. One of your friend tells you they paid to get in while another one cracked an exam. Just based on this information would you in your mind have higher bar to give credence to the smartness of the one who paid? They could be the smartest of all…but you likely want to see first, while you may accept the other person’s smartness at face value

But if I do follow you, I already find value in what you say. Your blue status will likely just surface more of you ...i just hope not too much to annoy me from unfollowing.

This is possibly the bargain I will have to make if I went blue

This is a conversation space in the Substack app that I set up exclusively for my subscribers — kind of like a group chat or live hangout. I’ll post short prompts, thoughts, and updates that come my way, and you can jump into the discussion.

I plan to use it for AMAs and anything else I can help with

To join our chat, you’ll need to download the Substack app (messages are sent via the app, not email). Turn on push notifications so you don’t miss a chance to join conversation as it happens.

How to get started

1. Download the app by clicking this link or the button below. Chat is only on iOS for now, but chat is coming to the Android app soon.

Get app

2. Open the app and tap the Chat icon. It looks like two bubbles in the bottom bar, and you’ll see a row for my chat inside.

3. That’s it! Jump into my thread to say hi, and if you have any issues, check out Substack’s FAQ.

Join chat

I’ve already started my first chat thread so come on over and say hello!

One problem that stumps many new PMs is measuring the impact of small features.

Eg: how do you justify any changes to orders screen, or making slight changes to the text in some obscure corner of product, or just informational changes .

What if you added small delight features like a fancy error message or a funny wait timer.

While you would LOVE to A/B test everything and see if these changes are positively affecting the core metric, unless you have a large number of users hitting those scenarios, you would not be able to get meaningful stats sig results.

Companies with billions of users can test even the minutest of things, eg Google allegedly tested 41 shades of blue , and Uber can test its custom fonts, but it’s unlikely that many companies will have the scale necessary to observe a stats sig result.

So how do you make a case for making those changes and if you do that, how do you measure these.

One way of course is to simply not measure the success (Or measure but do not expect). Think of these as paper cuts, its annoying but one won’t really kill you. The idea is that you go with what you deem is right and not get into analysis paralysis mode. You see a paper cut and fix it.

But relying too much on gut as two major disadvantages

1. Experience: Unfortunately developing a good “gut reaction” is a tricky problem and requires a lot of past experience to rely on. “Gut” is but a culmination of lot of data you have already seen

2. What is not measured is not rewarded: Unless you are running your own startup or have significant stake in one, you want to make sure you get recognised and rewarded for your work. If there is no way to measure something, there is no real way to recognise and reward it. All it may get you is a pat on the back . It is also deeply unsatisfying because you have NO idea if you are actually adding value
   
   See this absolutely hilarious clip from conan show (Start 6:07)
   

One simple way of measuring these changes is “constant holdback”

Whenever you roll out such an experiment, instead of rolling it out to 100% of users, roll it out to only 95% of your users. Do the same for all other small changes, with the same 5% always excluded.

As experiments pile up, the effect of multiple paper cuts being fixed would start showing up .

You would potentially be able to see the holdback group having meaningfully different(worse) metrics than the rest of the group..

But how do you know which experiment worked:

That is the point, you potentially will not. You will see cumulative effects and not specific.

What if some experiments are actually harmful

The idea is not to have all super positive experiments but to know if you have been directionally right. The trick is to pick mundane obvious changes that you need to do rather than using this for large features. If you are directionally right, you could eventually see a nice bump.

Can I do this for large new features

If you can do control vs treatment A/B test for any feature I would advise you to not use this holdback method as the primary means of testing. You do not want to have large experiments in the mix that can completely change the general direction of overall result

But I would argue that this hold back is useful even for large experiments once you have tested them . Post your A/B experiment you can think about a holdback for large features as well. The reason is that not all changes are plain additive. Eg: If you see your conversions go up by 2% via one feature and 3% via another, on long term its not necessary that overall it would be 5% . A constant holdback would tell you the cumulative effect of all the large changes you may have done

Do not forget to delete the holdback eventually so that all your users can see the same improved experience.

(image created using Dall E)

A lot has happened on Twitter vs Musk saga since my last letter

I had written about how a lot of people were misreading twitter’s 5% claim (Potentially deliberately) , and how twitter claimed 5% of it’s mDAU users were spam and NOT all users. A good chart to summarise it is below.

Very recently, Twitter’s Ex Head of Security became a whistleblower(Source) and revealed a lot of details about its security practices and also Spam accounting.

Keeping the security bits asides, it seems that even the whistleblower, who typically would be very antagonist to twitter, more or less confirmed that what twitter was reporting all along was correct.

SPAM in mDAUs are ONLY the users who slip through their existing spam filters

Twitter, Zatko’s disclosure claims, actually considers bots to be a part of a category of millions of “non-monetizable” users that it does not report. The 5% bots figure that Twitter shares publicly is essentially an estimate, based on human review, of the number of bots that slip through into the company’s automated count of monetizable daily active users, the disclosure states. So while Twitter’s 5% of mDAU bots figure may be useful in indicating to advertisers the number of fake accounts that might see but be unable to interact with their ads, the disclosure alleges that it does not reflect the full scope of fake and spam accounts on the platform.

Executives are incentivized to avoid counting spam bots as mDAU, because mDAU is reported to advertisers, and advertisers use it to calculate the effectiveness of ads. If mDAU includes spam bots that do not click through ads to buy products, then advertisers conclude the ads are less effective, and might shift their ad spending away from Twitter to other platforms with higher perceived effectiveness.

However there are many millions of active accounts that are not considered “mDAU,” either because they are spam bots, or because Twitter does not believe it can monetize them. These millions of non-mDAU accounts are part of the median user’s experience on the platform. And for this vast set of non-mDAU active accounts, Musk is correct: Twitter executives have little or no personal incentive to accurately “detect” or measure the prevalence of spam bots.

Twitter announced a new, proprietary, opaque metric they called “mDAU” or
“Monetizable Daily Active Users,” defined as valid user accounts that might click through ads and actually buy a product. 19 From Twitter’s perspective, “mDAU” was an improvement because it could internally define the mDAU formula, and thereby report numbers that would reassure shareholders and advertisers. Executives’ bonuses (which can exceed $10 million) are tied to growing mDAU.

Unless you’re a Twitter engineer responsible for calculating mDAU, you probably wouldn’t know what Agrawal is talking about. He is not saying that fewer than 5% of all accounts on the platform are spam. He’s saying, more or less, that Twitter starts with all the accounts on the platform, tries to automatically put all the human accounts that could be convinced by advertisers to buy products (but no spam accounts) into mDAU, and then uses humans to estimate the error rate of spam accounts that nevertheless slip through into mDAU. And naturally, Twitter “can’t share” its special sauce for determining mDAU.

Even though it’s written in a very antagonist fashion, what Zatko is saying should be music to Advertisers and twitter BD teams.

It says that Twitter took great care in making sure ads were not shown to suspected fake users and voluntarily removed them from its monetizable pool. It further claims that Exec comp was tied to increasing this specific metric rather than the “Vanity metric” DAU.

This is a GOOD thing. Anyone who works in Ad tech or marketing would tell you that.

Sure twitter can do more to fight spam, sure spam makes user experience worse, but there is currently no evidence that twitter lied in it’s SEC filing..

He does reveal a lot of things that can potentially give Musk an escape hatch from the deal , or musk would need to prove that there were issues with tagging spam in mDAU

Is mDAU really that bad?

There is another aspect that I now see some people raising: That mDAU is a “made up” metric and has no validity. While you can have your opinions on this, I disagree on this assertion and this is what I really want to talk about in this pos

There are three premises to this argument that I hope to address in this article

Premise 1: Twitter is using a non- standard metric

Premise 2: It is a bad metric

Premise 3: Twitter is not measuring it right

Premise 1: Twitter is using a non- standard metric

Let’s be clear, there is NO standard usage metric that you are supposed to report . The government mandates public companies disclose financial data in certain format, but not how a company measures usage.
Every company decides what are the most important measures for it and reports them. Even the basic metric like what is a defined as “Active” can vary from company to company based on it’s footprints

Eg: Snapchat only looks at people who opened their app(Annual report)

We define a DAU as a registered Snapchat user who opens the Snapchat application at least once during a defined 24-
hour period.”

From snapchat annual report

Whereas Pinterest(Annual report) accounts for all kinds of actions including web visits

We define a monthly active user as an authenticated Pinterest user who visits our website,
opens our mobile application or interacts with Pinterest through one of our browser or site extensions, such as the
Save button, at least once during the 30-day period ending on the date of measurement

From Pinterests annual report

Companies, especially large ones, routinely create their own combination metrics that make most sense to them.

Eg: You won’t care much about how many times Uber app was opened, you care about how many users actually took a trip.

Uber has it’s own metric called Monthly Active Platform consumer

Monthly Active Platform Consumers. MAPCs is the number of unique consumers who completed a Mobility or New Mobility ride or received a Delivery order on our platform at least once in a given month, averaged over each month in the quarter. While a unique consumer can use multiple product offerings on our platform in a given month, that unique consumer is counted as only one MAPC. We use MAPCs to assess the adoption of our platform and frequency of transactions, which are key factors in our penetration of the countries in which we operate.

From Uber’s annual report

Similarly Facebook, which is a direct competitor of twitter, is also introducing a new metric called Daily Active People (Annual report)

Family metrics represent our estimates of the number of unique people using at least one of Facebook,Instagram, Messenger, and WhatsApp

From Facebook’s annual report

Premise 2: It is a BAD metric

In short: Twitter’s mDAU metric is the number of people who it can show ad to . Twitter removes potentially suspected bots, spam accounts, and also accounts only posting via APIs etc from mDAU count(See more details)

This is an ABSOLUTELY GOLD metric. If you are in the business of selling ads, making sure you only show ads to real humans is an extremely important measure.

Every company would have some measure of this. Twitter just chose to disclose this and make this their key metric. It is a strong signal that they are in the business of selling Ads.

Twitter can absolutely disclose and measure overall spam accounts, but that does not take away the validity of the mDAU metric.

I also keep hearing that twitter removes Bots and SPAM from it’s calculations of mDAU, do you know who else does that ? Pinterest . Here is a direct quote from their annual report

We regularly deactivate false, spam and malicious automation accounts that violate our terms of service, and
exclude these users from the calculation of our MAU metrics;

From Pinterest’s annual report

Pinterests Daily Active user seems functionally equivalent to twitters monetisable Daily active users. I see nothing inherently bad in this metric.

Another point to note: Facebook and Snapchat seem to not take out spam accounts from their Daily active user count. Facebook does mention how many active users it suspects to be spam, but I did not find anything related to that in Snapchat’s filings.

There is no consistency or rule on what is a”Daily active user”

Premise 3: Twitter is not measuring it right

Here is the process twitter follows, it has a bunch of AI / ML algos that automatically remove as many accounts as possible that. it suspects are spam . Twitter than samples 100 accounts per day(9000/quarter) from the rest of the accounts and have manual reviewers rate if these accounts were SPAM or not(Triple checked I read somewhere).

They do this everyday to get a trend and find that approx number of spam users that pass through their filters is <5%.

There are three objections I hear in this regard

Objection 1: The Sample size is too small

Statistical significance is not related too much to sample size, but rather to sample selection. Typically a sample size of 100 can give great representative results for a large population, twitter is doing 9000(over a quarter).

Eg: CNN did 2020 presidential election exit poll with a sample size of just 15,590

Objection 2: They do manual review

Of course they do. They already used their AI/ ML algos to filter out all spam they could and now manual is the last step. Infact, even facebook uses manual reviewers to tag spam

Facebook defines them as “Violating accounts” (Another non standardised name)

We define “violating” accounts as accounts which we believe are intended to be used for purposes that violate our terms of service, including bots and spam.

From meta’s annual report

It goes on to explain how they determine if an account is violating

Such estimation is based on an internal review of a limited sample of accounts, and we apply significant judgment in making this determination. For example, we look for account information and behaviors associated with Facebook and Instagram accounts that appear to be inauthentic to the reviewers

From meta’s annual report

Objection 3: it can’t be 5%

Do you know how much TOTAL spam facebook claims it has? 3%. NO Kidding

we estimated that approximately 3% of our worldwide MAP consisted solely of violating accounts”

From meta’s annual report

Assuming there is no lying, if facebook can have only 3% of all its users as SPAM, twitter’s 5% SPAM after SPAM filters does not seem off.

I do suspect though that I may have missed something and facebook also removes suspected SPAM accounts before calculating “Violating accounts”, which makes it potentially functionally equivalent to mDAU of twitter

Also remember, mDAU is NOT a user facing metric.Your own experience is immaterial. Twitter can have 20% SPAM and still have only 5% SPAM in mDAU.

So NO mDAU is not really that non standard, nor is it specifically bad , nor does it seem twitter’s revealed methodology is anything shady.

There can obviously be something deeply wrong with twitters count if they are hiding something, but I am unable to see any info about that yet.

Want to know about a real BAD metric that’s super popular and is also stated in annual reports? Read why NPS scores are useless

Only problem, Twitter did not exactly make that claim , and as usual the tech media decided to ignore that, deliberately I believe(more on that later in article) .

The Claim

Lets first look at the filing that everyone keeps referring to
Here is the exact line from the filing

The actual claim is

Average of false or spam accounts during the fourth quarter of 2021 represented fewer than 5% of our mDAU during the quarter.

Lets define the terms:

DAU: Daily Active user
mDAU: Monetizable daily active user

The “m” is super important. So what is the difference? While I would love to think that it’s possibly industry specific terminology that most people do not get, twitter in its annual report actually defines for anyone who bothers to read.

We define mDAU as people, organizations, or other accounts who logged in or were otherwise authenticated and accessed Twitter on any given day through twitter.com, Twitter applications that are able to show ads, or paid Twitter products, including subscriptions

So what twitter is saying is that of the number of people who they could have shown Ads to , only 5% of them were SPAM as per their estimates.

This implies you will have to remove any accounts that tweet using systems where No ads can be shown.

For eg, its likely that you would not see an Ad if you used an API to post a tweet, and this may extend to third party clients which allow you to post. Eg: I sometimes use roam(My notes app) to directly post.

Twitter APIs allow you post 200 tweets in a span of 15 minutes

This changes a lot

• Lots of bots and spams would be using automated scripts and APIs to post. They would never be on a surface where they can be shown ads, hence Non Monetisable. Thy are not counted

• Real users tweeting using certain clients (Or automated scripts like IFTTT) may not be counted

• Any account which is spam or even likely spam may be tagged by ad engine as such, and removed from potential monetisation and hence not counted. Twitter even mentions that in their filing in the same para as the 5% claim

After we determine an account is spam, malicious automation, or fake, we stop counting it in our mDAU, or other related metrics

So possibly a large swatch of accounts that may be labeled as potentially spam and fake never get to see an Ad, and hence not counted.

Even Twitter’s CEO has stressed that its mDAU

Not every potential spam account is deleted , possibly because there can be lot of false positives . Lot of real people behave like bots and the ad engine may have stricter rules

Fun thought exercise: If you behave like a bot, do you get ad free twitter?

A good visualisation of this would be something like this

So fake accounts on twitter could be 20% or even 50%, if they are not being monetised, it’s not counted.

The main claim in some sense is : If an advertiser spends money to reach users on twitter, only 5% of those users would be Fake.

This is an advertiser facing metric and not a user facing one. Your own experience is not what is being measured

Now coming back to how it gets reported. Remember the screenshot of reuters I shared above? In the the sub heading they do decide to make that distinction, indicating that they know this difference but chose to NOT talk about it in main heading.

This is repeated across many articles across various tech media sites. Either they ignore it and assume mDAU =DAU(which is incompetence) ,or hide it in text which I think is not very ethical.

This distinction is so important that it needs to be called out in the MAIN heading

Some examples of Media reporting:

Reuters

Bloomberg

Forbes

Business Insider

So does twitter not have Bot problem?

Not exactly. There are 5% fake users on a platform is very different from “of the people who can be shown ads, only 5% are Fake”.
The data needed to verify this claim is

• Who was monetised

• Take a sample of these monetised users

• Define and agree on the principles if what is SPAM/ Fake account

• See what %age of these users fit that definition

This. is why its almost impossible to verify this claim without having access to twitters internal systems.

What percentage of SPAM accounts exists severely affects users and have a negative effect on user experience. This absolutely needs to be addressed, but the claim twitter is making is not about a user facing metric but rather an advertiser facing.

The big question that needs an answer is : What percentage of twitters daily active users in monetisable.

Its very much possible that twitter is lying, or maybe they count every DAU as monetisable, maybe their SPAM engines are too lenient but we need internal data to know that .

I for one do not suspect twitter doing anything shady .

So do not blindly believe the headlines, and develop a lot more skepticism

First a note on digital art: Thanks to my friend Manas, I have finally bought the procreate app and restarted doodling / painting. One of the things that hit me most was how damn easy it was. It almost felt like cheating. It is the exact same kind of feeling I got when i discovered loop music which helped me compose background score of many of my poetry compositions

Whenever barriers of entry into art are lowered we see many more people join and many new type of art emerge. Loop music made scores of musicians more productive and also increased the sheer diversity of sounds we all are now exposed to, I assume similar things are happening in digital art world as well.

I hope soon it will happen in no code space also, when it almost feels like cheating. At the moment I do not feel that way, which maybe because I have worked in tech long enough so my bar for “magic” is higher.

Back to NFT: So after creating a simple art piece following a few tutorials, I minted it on Matic chain. My aim was simply to understand the process and was blown away by how easy everything has become and how it can already do so much. For eg: I actually set a 10% royalty, so whenever my NFT is resold on opensea, I make 10% of the proceeds in perpetuity automatically. No clearing authority or anything

A big shout out to Shobhit for teaching me how to mint it and also Buying the NFT :D. Its also transparent so you can actually see how much it was sold for :)
Here is a video from shobhit explaining how to mint your NFT (Note the real action is on ether chain, not Matic, but ether is super expensive at the moment )

Angel Investments

Soooooooooo…..this year I finally started investing in some incredible founders and one of the companies I invested in just announced the funding round, hence I can talk about it :)

I invested in location analytics startup called Locale.ai . What I loved about the company was their focus on solving a very specific hard problem. I anyways am a fan of SAS and their demo was incredible. Also, I am super bullish on the founders Aditi and Rishab. You can try their demo here . They are also hiring, so reach out to Adit if you are interested

I hope to be able to talk about the other incredible startups I got a chance to invest in soon.

If someone suggest insurance+ investment combo opportunity …RUN:
NEVER EVER mix insurance and investments. It makes no financial sense. So many of us are stuck paying premiums for junk polices like Jeevan Anand, just because our neighbourhood uncle convinced our parents that it is a good deal.
Should we start a “Jeevan Anand Peedit Sammelan”

Remember Benchmark:
Whenever someone talks about “great returns” always benchmark it against its peer. Eg in last one NIFTY 50 gave 42% returns. Almost everything did well post the covid market crash. I am pretty sure some agents are trying to sell you policies quoting an awesome 20% return last year . Ideally ignore the last year anyways

Fee only financial planners ONLY .
Avoid your bank’s wealth managers at all costs. The incentive of anyone who makes money via your investing is not aligned. More often than not they would try and sell you what gets them maximum commission rather than be beneficial to you.

Direct mutual funds only: There is no real advantage of giving commission to the brokerage. Direct mutual funds not only give better returns, they are now super accessible. Just go open an account with Zerodha (referral link) and start investing there. I liquidated all my MF holdings after they crossed the threshold of exit load and moved to Zerodha

No point buying an apartment:
I personally see no point plonking in a huge sum of money into an apartment, especially in places like Bangalore.
– The rise in real estate is more or less correlated with stock market growth
– Rental yield is just 2-3%.
– The houses you want to stay would not have the same EMI as rent ( as some ppl claim). You just end up paying “Rent” to the banks, and its super expensive and you are locked in
– There is also cost of non mobility

The only reason to buy a house is emotional which obviously cannot be priced in. If owning a house helps you feel good and has a positive effect on your self image, thats your call. To me at the moment it makes no sense. I may eventually buy when I have enough spare.

Book I recommend

I typically recommend Lets Talk money  by Monika Halan . It is simple, and good enough for most people. Personal finance should be simple, you are not managing a hedge fund

More Book recommendations

Also see: My US stock investment ( I invest from India)

Will start sending out some infrequent recommendations on email as well. I hope you find these valuable.
PS: The links below are referral links

PPS: Amazon is running a Prime day sale and it might be a great time to load up. Apparently they are also offering a 4 month subscription of Audible for onlyRs 2. You can get 4 audiobooks for free, which I think is terrific deal.

Product management and Other Non Fiction

Cracking the PM interview: I recommend this book even if you are not interviewing. I find this a good primer to what a product management role looks like

Traction: How Any Startup Can Achieve Explosive Customer Growth: This was recommended by the CEO of a previous company and I have found it super helpful to think about interesting ideas and strategies to grow your startup

Inspired: How to Create Tech Products Customers Love:Pretty good commentary on how large product companies build products and also the role of product manager.
Tip:The hardcopy seems pretty expensive, but its available on audible, if you signup on audible you essentially get it for FREE. I also took the audio copy for the same reason

HBR’s 10 Must reads on Communication: Product Managers need to be great communicators. This collection of essays by HBR is a great read and can be referenced periodically

The Hard Thing about Hard Thing: Building a Business When There are No Easy Answers : A fascinating read on how building a company is not an easy thing. I particularly loved the “Peace time CEO” and “War time CEO” analogy. Since Product managers are in centre of the storm many times, I think this book gives a great perspective which can be handy in your professional life
Also this is written by Ben Horowitz, cofounder of Andreessen Horowitz. I am planning to read his new book What you do is who you are soon

On Writing Well: The Classic Guide to Writing Nonfiction: A great guide on how to write well. Written communication is super important for product managers and it pays to hone this craft

Elements of Style: Another book on writing. This book is fairly technical

Design of everyday things : A classic on basics of design. As product managers it is important to understand and appreciate the design process and this book is a good primer to help you understand what is design.

Zero to One: Notes on Start Ups, or How to Build the Future: A book on startups and creating value by none other than Peter Thiel. Its interesting to see his take on “Monopoly can be good”

Super Thinking: Upgrade Your Reasoning and Make Better Decisions with Mental Models : A really good handy book on mental models. I saw this recommendation in a talk by Shreyas Doshi

On Writing: A Memoir of the Craft Kindle Edition: This is a part memoir, part writing instruction by none other than Stephen King. It inspired me to pick up a pen and write more

Checklist manifesto: The power of a simple checklist. Post this book you would have new found respect for something so simple

Dale Carnegie:Art of public speaking: I was incredibly lucky to have come across Dale’s work early in my career . Becoming a better speaker will serve you well irrespective of what work you do.

Science

The order of time : Narrated by Benedict Cumberbatch :Did you know even time is possibly quantised? It does not flow, but rather takes steps. Who better to talk about TIME than Dr Strange himself. (you can get this and 4 more audio book free if you signup for audible during this sale).

Something deeply hidden: Everything from how did space time come about, to how does multiverse theory fit nicely, to how we can view the world as a single wave function. I listen to an audio version of this book often just before bed. It’s fascinating and much more updated to reflect the current understanding of the universe

Brain : How our experiences shape the brain, and how the brain shapes us

Sapiens: A brief history of humankind : A thrilling read about how humanity evolved. My big takeaways were about how our society functions on multiple figments of imaginations and social constructs. Money, bank, government, and even morality don’t really exist, except in the shared imagination of all of humanity

A Brief History Of Time: From Big Bang To Black Holes : When one of the most brilliant minds of our times, Stephen Hawking, explains science to common folks, you listen.

Science Fiction

Foundation Series: Issac Asimov’s classic, this series is considered one of the best works in science fiction. It is an intricate multi generational plot that shows the birth, death, and rebirth of a civilisation with science and tech being the background of the story rather than the centre piece

Remembrance of Earth’s Past: I was introduced to the works of Cixin Liu recently and have become an enormous fan. He is like a living legend and no one has grabbed my attention like his work except the work of great Asimov. His works pushes the boundary of what’s possible. Super heavy on tech and science. It consists of three books
Three body Problem, The dark Forest, Deaths End

The Ultimate Hitchhikers Guide to Galaxy: Smart and funny. Who knew science fiction can be this hilarious.

Are you interested in Book recommendations? I send out handpicked Book recommendations occasionally . You can signup to BooksByAll Substack here for free.

They say, the key is to not drink the Kool Aid.

While I speak from a more Product manager’s perspective, this is generally true for everyone who wants to be intellectually curious

It’s easy to be be a believer, it gives you sense of purpose, a sense of comfort, but if you really want to succeed, you need to be a bit of a skeptic.

And skepticism can be taught. Here is a simple trick I suggest

Developing a habit of skepticism

Whenever you read or listen to something interesting, something that catches your eye, especially something that makes claims: Think about one small fact that you can verify. I typically add a small note “really?” in my personal notes.

Go ahead and try to see if you can substantiate that. It could be a very simple thing: Eg someone says that a new study says that Covid vaccine is very effective, you can just check if the study exists and it makes that claim. It could be even simpler than that: Eg a startup says that their market is all tax payers in india and that is Y Million people. Just try ad find that data, is that accurate.

Slowly you start moving to questioning the interpretation on those facts. Eg: In the covid vaccination is “good” case above, you could now try and substantiate what is good. The research report may say 80% efficacy. Is 80% good? How does it compare to other vaccines. Are there any specific things that have been missed? Which age group, which demographic, which variant?

Eventually you start questioning the whole premise of the argument itself.

At the highest level you move to the very motivations driving the argument.

Most things you evaluate would be correct, but that is not the point. You are not trying to find malice, but just building a muscle for questioning.

With enough practice you would start seeing a pattern. You will get a “gut” for understanding what to check and verify.

Every industry, and also every individual has a pattern of what they tend to overlook or what they exaggerate. It could be personal bias or just a generally accepted “industry practice” (see Uselessness of NPS score article as an example of how a generally accepted industry practice is not necessarily accurate)

With this you are not trying to be cynical, but just being skeptics.

A very good exercise might even be to treat this very article with skepticism

• Have I defined skepticism right

• Can skepticism be taught, or is it just genetic

• Who said PMs need to be skeptics. Is there some kind of qualitative or quantitative evidence to support that claim?

After writing this article, a perfect opportunity to demonstrate this came about. I started seeing some WhatsApp forwards and tweets talking about how upto 40% apple workers intend to leave for lack of full remote, or 90% apple employees want indefinite remote.

While I am all for flexibility and do believe that full remote is here to stay, the 40% / 90% number seemed way too high.

So I decided to look a bit deeper. Thankfully one of the newspaper itself posted all the details including their own skepticism

• This data does exist. It was collected from employee survey done at Apple

• It was done by Apple Employees themselves

• It was done in a slack group specifically meant for people invested in remote work . DUH!!!

Without even trying too hard you can see that only 36% of employees in a group specifically meant about remote work spoke about resigning.

If I was teaching a class on bias, I would use this as a perfect example.

You can draw absolutely NO conclusion about what apple employees want in aggregate from this. While I do give points to media sites for publishing the survey details, I hold them accountable for publishing it in the first place knowing fully well that this data has no validity.

It leads to absurd headlines and unnecessary conclusions amongst people who trust them.

It’s like me doing a survey in a “Board game lover” internal group about how important it is to have board games in the break room, I may get 90% Yes. That does not mean 90% people in my company want boardgames in the break room.

No company is going to lose 40% employees just because the do not offer full remote work. Ironically, the most accurate data about the pulse of their organisation might be with the company itself. The company’s survival depends on it. No one would risk losing 40% of employees.

You can obviously dig further and look at specific questions and see if these questions had inherent bias already. Surveys are not that easy and results can vary widely based on how you ask a question(More on this later).

Motivation: Now you can ask, why did these news sites publish these results knowing fully well that they are widely inaccurate and biased. What is the motivation:

• They are judged by clicks

• Appealing to bias of people leads to more sharing

• Apple is the current favourite punching bag
  

As some of you already know, I am LONG Twitter(see my US stock holding pattern ) and feel that it has just started to scratch the surface of what is possible.
I have also been thinking about what twitter is upto and where it can potentially go. Here are some early thoughts

So whats new

Recently twitter announced a slew of new features including

• Spaces : Clubhouse competitor

• Subscriptions: Subscribe for

  • Exclusive tweets / content

  • Newsletter

  • Badge

  • Community

In January twitter also acquired revue, a newsletter company to let twitter users start newsletters

They have also reopened verification, so you can get that sweet sweet blue tick

Twitter has also started to hire aggressively in India, and their tech office in Bangalore has multiple open roles including Staff level.

Staff level roles are generally needed when you are willing to start large teams around that person, clearly indicating that Twitter has all the intentions to grow.

This definitely is a good sign because it shows that tech is finally becoming a big enough bottleneck. I expect many new features coming out of Twitter’s stable

Where is it going

I believe twitter is unleashing the beast with the power of its platform and could eventually go into Identity and Social capital management. Here are some of my early thoughts

Twitter as Identity Management

Once twitter starts mass verifying users, it would become perhaps one of the largest consumer facing identity management platform in the world. It would be bigger than many governments and will come with added social capital.

When someone says they are verified, you not only know who they are, but can also go and check out their profile.

There is a massive need for user identity verification especially in situations involving people to people contact. Eg Uber users in brazil need to validate via CPF(national ID ) if they want to pay via cash. This helps them establish trust. Think about using twitter globally

Other user cases could be

• Is the person sending me the email the same as they claim to be

• Is the person bidding for my furniture real

A lot of startups bootstrap on twitter network, eg: Substack shows you who amongst the people you follow on twitter has a substack. My own Substack had subscribers even before I sent my first newsletter

Imagine what could be done with high trust authentication.

Twitter as Social Capital Management

Whenever any influencer or content creator opens up another channel (Eg a new newsletter, a youtube channel, a blog) they end up recreating the entire network again. The need to gain the trust of not just the followers but also the platform.

While surely there are massive advantages of existing audience which follows the creator, there is always a huge leakage and the creator is subject to very different rules on each.

Eg: Even if you are extremely popular on twitter, you cannot take that social capital to say a mailchimp to send your newsletter. You need to “warm up” the system, hope your audience opens it, prey that spam filters don’t flag you, and if you have a shared IP hope all other providers are not doing something shady.

Mailchimp has no context of who you are or how you built your audience. There is no social capital associated with you.

But with twitter this could change. Twitter could use your social capital of one platform (say twitter) to jump start you on another (eg newsletter). It can use your social capital for more than just growth hack.

Will it matter to small time creators

While small social capital on one mode of communication may not matter to people, a multi modal platform may matter in aggregate.

The biggest advantage of this is that as people start deriving more value from the system, the incentive for bad behaviour becomes less You also become super cautious if you verify at any place using twitter because you don’t want to mess with your social credit.

Gotchas to watch out for
It needs to be debated if it would be wise for a single private party to wield such power. Would your twitter identity become more important than your local identity?

Some interesting ideas I think twitter would / should do

• More control on tweet embedding: Let me generate a id that any media agency that wants to embed my tweet needs. I can then charge for my tweets

• Transfer tweet ownership: Currently if your tweet becomes popular, people reach out to you to add their product / business as a reply. This helps people make some money. It would be interesting to see if I could just pass on the tweet ownership to someone else and they would be free to add any reply to the main thread

• NFT: While some startups are attempting this. Every tweet could be an NFT and that can drive the tweet embedding and ownership ideas as well.

• Tweet Ads: Every time a tweet is embedded on an external site, twitter could show and ad and share revenue with the tweeter directly

• Mail client: While twitter knows about the social capital, it still is hindered by the spam rules of other mail client providers. I suspect they may start their own

• Influencer Management / Tracking / smart contracts: Imagine you hire an influencer for reaching X million audience. The entire deal is done on twitter where twitter is responsible for measurement of success (which they already do). As soon as the success criteria is met, the influencer is paid . This operation could be controlled via ethereum smart contracts. This makes the entire system seamless and painless

You may have noticed I did not talk about tweet edits.
Read why I don’t think twitter would ever, or should ever allow tweet edits

Also read
The Uselessness of Net Promoter Score:

While NPS(Net promoter score) maybe an extremely popular metric to understand customer loyalty, there is little evidence to support it’s claims.In this article I detail out everything from history of NPS, to uses, to issues

This is a very longing post 3400+ words and counting so

Sometime back I had tweeted a thread about how I dislike NPS score as a measure of success. It led to a fair amount of discussion and debate. Hence, I decided to dig a bit deeper and write a longish post about it. While I have tried giving it some structure, each section is more or less self contained. You can directly jump into a specific section if you are already aware . I have tried to give enough context wherever possible.

I have also linked to sources wherever applicable, so feel free to follow them and do your own due diligence when in doubt.

Disclaimer: I may add more details and address any specific questions and criticisms that may come my way. Please do let me know if I misrepresented or missed something. More updates as they come by would be on my blog

What is Net Promoter Score AKA NPS

Net promoter score is a widely used measure of customer loyalty today. It’s claim to fame is its utter simplicity. It can measure customer loyalty with just 1 question.

To calculate Net Promoter score you ask a statistically significant number of your customers to answer a single question

How likely is it that you would recommend [brand or company X] to a friend or colleague?

• Ask them to select from a scale of 1-10(some orgs use a slightly different scale but 1-10 is the most widely used), where 1 means not likely at all, 10 means very likely and 5 means neutral.

• Anyone who choses 1-6 is considered a Detractor

• Anyone who choses 9-10 is a Promoter

• Net Promoter Score= %age of Promoters – %age of detractors

The basic claim of NPS is that it can reliably measure customer loyalty and if the company focuses its efforts to increase NPS, it can lead to more healthy growth.

This two specific claims are important to keep in mind
1) Reliably measure loyalty (better than other scores)
2) Correlated to company growth (see more details in NPS Origin story sec)

NPS origin story

While internet is filled with how to use NPS, when to use NPS, and why to use NPS, before we get to all those questions it is necessary to understand how NPS even came into picture.

The origin story of anything reveals a lot about the motivations without the burden of muddled history between then and now.

NPS score was invented by Fred Reichheld who was a consultant at Bain and company . It was introduced to the world in this HBR article

The basic idea came about when they looked at a car company use a very simple method to increase customer loyalty. The company, Enterprise Rent-A-Car, simply asked people two questions
– Quality of rental experience
– Likelihood you would rent again

The company then counted only those customers who gave it the highest scores on both the questions. All their outlets were then asked to optimize for this specific score. It was believed that this would inspire the sales agents to be better and increase customer loyalty.

Fred wanted to make this system much more simpler and see if this could be brought down to just 1 question

The interesting point to note here was that the intent was not to find a great predictors of company growth or loyalty, rather to find one question. The Aim itself was simplicity

Process of finding the One Question

1. 20 questions were created on the Loyalty Acid test survey

2. Test was administered to customers in following industries

   • Financial

   • Cable and telephony

   • Personal Computers

   • ecommerce

   • auto insurance

   • Internet service providers

3. Then they asked each participant to tell about a specific instance when they actually referred the company to someone. If this was not available they waited 6-12 months and asked again. This data of about 4000 users was enough to create 14 case studies which established a link between survey response and actual referral

4. Result

   • The top-ranking question was far and away the most effective across industries:

     • How likely is it that you would recommend [company X] to a friend or colleague?

   • Two questions were effective predictors in certain industries:

     • How strongly do you agree that [company X] deserves your loyalty?

     • How likely is it that you will continue to purchase products/services from [company X]?

   • Other questions, while useful in a particular industry, had little general applicability:

     • How strongly do you agree that [company X] sets the standard for excellence in its industry?

     • How strongly do you agree that [company X] makes it easy for you to do business with it?

     • If you were selecting a similar provider for the first time, how likely is it that you would you choose [company X]?

     • How strongly do you agree that [company X] creates innovative solutions that make your life easier?

     • How satisfied are you with [company X’s] overall performance?

Link between NPS score and company growth

Then they tried to find correlation of NPS score of customers with the actual company growth

• In airlines a strong correlation existed between “Would recommend” question and average company growth

• Similar results existed in rental car business

• “Would recommend” was irrelevant for database software or computer systems as people had limited choice, and senior execs who made the choice were not part of the people surveyed. For such industries “Sets standards of excellence” and “deserves your loyalty” were far more predictive

• NPS was also not a predictor for Local telephone and cable TV company growth because they were near monopolies. Their growth was determined by how fast the population in their area increased

Who uses NPS today

Pretty much every one. As of 2020 2/3 of fortune 1000 companies seem to use a version of NPS. One simple experiment would be to search for the term “How likely” in your Inbox

Good things about NPS

It is very simple to measure and benchmark .

• Its a single question and is used by multiple industry players to benchmark against competition and internally

• Its easier to digest at almost all levels of abstraction

High completion rate

With users being inundated with all kinds of brands seeking their attention, it is much more easier to get them to answer 1 question rather than multiple. Infact in the paper “Assessing treatment outcomes using a single question” where they did an NPS of patients, they found that the NPS question consistently had the highest completion rate (96.5%). I would also now assume that it has become so common that users almost expect this question and are willing to answer

It defines loyalty in interesting way:

While loyalty may traditionally be defined by retention, LTV, and other metrics , it can miss out on word of mouth. NPS attempts to target that specifically by a bit lose but interesting way

Customer Loyalty Definition in Original NPS system

• Customer loyalty can be defined as customers willingness to stick to certain provider even if they are not providing the best possible rate in a particular transaction. Think if this like : ” Sure you may be charging me more today, but I know you have done great work in the past and generally give me good rate so I will stick to you even though cheaper options may be available”
  

• Customer loyalty is also more than just retention because some people maybe retained just because they cannot move out due to inertia, or exit barriers. Eg : Monopoly players , or prepaid plans
  

• Loyal customers may also not be repeat purchasers, eg when they outgrew that service. Eg: You may no longer buy a pulsar bike because you no longer drive a bike, but you would recommend it to your nephew when he is considering one.

NPS claims and how they measure up

NPS is a slightly obtuse metric because instead of asking if people are satisfied with the product or service, we are asking if they would recommend it to someone else. It’s not exactly a measure of a customers own experience with the brand.

If you are introducing a new kind of measurement it needs to be better at something than the existing systems. It either helps you uncover a specific issue, or measure something unique.

Survey metrics also are predictors / proxies of some tangible business outcomes such as churn, growth, complaints, etc. A metrics with no business outcome is plainly a vanity metric.

So lets deep dive into if NPS measures up

NPS as a better predictor of growth

Let’s look at the claims made about NPS in its original research. There are multiple leaps of faith in it. The way I read the original article is:

• Answer to NPS question seems to be the highest correlated among other questions to actual referrals in some industries

• Higher NPS seems to be correlated to higher growth rate irrespective of company size

Using this above methodology claims have been made that NPS is the best predictor company growth. The big issue with this is that even in the original article there was no real comparison of correlation between company growth with NPS vs other survey methods.

Also even though the question seems to talk about loyalty in a very loopy fashion, it actually does not make a claim about it. There may or may not be no correlation between NPS and user retention

This research was not even reproducible

It is not reproducible

This is perhaps the BIGGEST issue with Net Promoter score. The biggest claim with NPS was it is the single best predictor of growth, but this 2007 paper found no support for that claim when they tried to replicate the same study that Reichheld did.

Not surprisingly, they found that NPS performed as well or as poorly as using the customer satisfaction index to predict growth

There seems to be no real statistical backing to NPS, and as per the paper, even Reichheld acknowledges that

NPS as a tool to benchmark competition

A lot of literature outside talks about using NPS as a benchmark against competition, between different departments, different franchise etc. A lot of fanfare is made about how a company’s NPS is through the roof, which company in a specific industry has the highest NPS etc.

The problem with this is that this question has so many variables that its unfair to compare . It can never be an apples to apples comparison.

Instead of simply asking if customers are happy with the service, we ask “Would you recommend X to your friends and coworkers”. There are so many more variables to consider when trying to answer this question

• Do I think it’s worth it for my friend: Hobbies, cost, personal interest, my own closeness to the friend

• Do I even discuss this with my friends and coworkers

• I hate it, but my this specific friend may like it

• etc etc

More variables = more errors.

Eg: when NHS introduced NPS they found that only about 40% variations in NPS scores was explained by overall satisfaction whereas rest was explained by various other metrics such as: if the patient undertook hip replacement or knee replacement.

Even the NPS difference between patients who underwent Hip replacement(71) and knee replacement(49) were stark, making it impossible to benchmark them.

A bad action item for the hospital would be to target for same NPS across all services.

If a hospital cannot even benchmark within its own departments, it’s useless to try and benchmark to other hospitals.

It’s also very dependent on services availed and demographics of the user . Eg: when they compare NPS of Uber vs Ola, they fail to talk about if it’s the exact same mix of users or not.
Did the users take similar number of trips, same kind of vehicles , pool vs non pool etc. Without that, comparing NPS of brands Uber vs Ola is not really any benchmark, and potentially worse than just satisfaction surveys.
It’s not a worthy complication you are introducing. This is why brand NPS are not really benchmarks

For internal benchmarks, some companies, especially in ecommerce, go overboard and try to find NPS linked to each product and service, which again seems unnecessary.
It is no longer a measure of loyalty but just a feedback of the product, which may be better asked directly via ratings and reviews.

Eg see below Myntra trying to do NPS linked to a specific product. But even if I rate it low they may have no action item because they do not control the product itself. Lot many questions need to be asked to even understand my response.
A better question would have just been do you like the product, which not only would be direct but also feed their rating system.

NPS as a better loyalty metric

This is another way some people use NPS for . This is potentially because of the nature of question where it talks about referrals. Loyalty here is defined as user’s willingness to recommend.

NPS tries to force fit people into specific boxes of promoters and detractors ignoring any reasoning behind the users response

There are many reasons you would not recommend a specific product to someone. Just like in NPS article they mentioned that a loyal customer may not be a repeat customer because they outgrew the product, but would happily recommend it to someone who did not.

Using the same logic, someone who may be a loyal customer may not recommend it to friends who may not be the target customers. NPS would classify these loyal customers as detractors.

Loyalty is very fluid and detractors and promoters are not rigid boundaries as NPS tries to bucket them i

Your personally may hate the product but if you were to suggest a product to someone you would play a matchmaker role and take into consideration their individual needs and circumstances.

This 2019 survey found that 52% users who actively discourages others from using a brand also actively recommended it. You can be a promoter and detractor at the same time based on who you are talking to or how your last experience with the brand has been.

To make matters even more complicated, NPS is not even an accurate predictor of users own measurable behaviour such as repeat purchase, churn rates etc

The purest measure of loyalty in my opinion is customers actually spending money to buy your product. I criticise my bank a lot, but despite many many alternatives I have stuck with them for 15 yrs. By every definition, I am a loyal customer who they would want.

A research done by University of Cambridge with an asset heavy company in UK over three years found that NPS and actual user behaviour did not match

NPS as more “accurate” measure

Another argument for use of NPS is that its less susceptible to manipulation, but NPS has same pitfalls and anyone who owns an NPS goal can use the same old tactics to improve it.

Just like satisfaction surveys which can be manipulated, so can NPS. Simple techniques could be

• Asking the user to rate you after a good interaction. Eg: As soon as order is delivered, or a ticket resolved. At this time you are no longer trying to find and fix issues, you are simply trying to get that score. This technique may have an effect for Play store reviews and youtube videos where ratings and likes are a social signal to other users, they are counter productive for NPS , unless this NPS is being collected as a vanity metric. Eg: Pitch deck, Presentation to leadership

• Incentivising the user: Eg give your software product for a free trial and see almost every customer give you high ratings on NPS. It is meaningless and may have no correlation to your growth
  

I also did a very unscientific survey on Twitter and Linkedin to know if companies took NPS targets, and if the person responsible for the target also controlled stuff like when NPS was sent and how to pacify the user: Here are the results

As per the survey above ~30% respondents said their company has NPS goals and the owner of the goal optimises of things like when to send the survey, and in some cases even customer incentive .

You get what you optimise for and in this case my hypothesis is that system is designed to make the NPS go up not necessarily the loyalty

You get what you optimise for

This maybe the reason why companies with really high NPS also go bankrupt

NPS and other big misses

Its arbitrary and ignores all cultural nuances

There seems to be no clarity on why someone who says 6 vs 7 are in a different bucket while 5 vs 6 are not. Also no clarity why focussing only on difference between promoters and detractors matter. What if we just tried increasing average score?

It actually hides the actual improvements. Eg: movement of a large chunk of users from 1 to 5 has no effect on the NPS score.

It also ignores all cultural nuances. Eg: if you travel in Uber in US vs India, you may see a huge difference in your ratings. Anecdotally I have seen my ratings drop in india and rise in US. I presume there is a cultural difference here. In India low rating is 1* whereas in US its 4* .

I read a comment on some blog that put it well: NPS is just lots of numbers disguised as maths

It is more noise than signal

What do you do with NPS? One common theme is that you work towards increasing it by using it as a north star, but that is not a good reason to ask this question in the first place.
There is no evidence that it is better than working to optimise other tangible metrics.

It’s not a single question

While the whole USP is its single question, you invariable would need more information as soon as the users rate <7 , defeating the whole purpose of simplicity.

Loyalty is multidimensional

While NPS seems to acknowledge that loyalty is multi dimensional, it tries to collapse it into a single dimension of word of mouth.

Its probably not for your industry

This is less of NPS issue and more of marketers abusing NPS because of its perceived simplicity.

In the original paper, NPS was not found to be a predictor of growth in industries such as computer databases.

Remember the ONLY thing it was supposed to do was predict if you will grow, without that correlation the score is more or less useless.

Sales is complex and any industry with high inertia, top down decision making, and monopolistic players NPS is not even applicable. This makes me wonder why so many startups are obsessed with it.

It’s also possible that NPS should not even be a goal.

Eg in the NHS paper I referred to, difference in NPS among patients was not due to actual patient care and recovery. Perhaps NPS is not even a measure for hospitals.

Final Thoughts

NPS seems to be a arbitrary score with little statistical backing. It is not even be valid for many industries .
While it can be used as a tool in your armour of many other signals, over reliance on this for making decisions is not prudent.

NPS is popular perhaps because it is simple, but this reminds me of the phenomenon of Bikeshedding .

Bikeshedding: If a committee were to design a nuclear power plant, they may spend far more time than necessary to discuss the bike sheds, its color, its position, and its capacity . The reason for this is that bike sheds are easy to design and everyone can have an opinion on it.
In corporate we sometimes spend a lot of time on bikeshedding activities just because our minds automatically go towards simplicity first.

NPS to me sometimes sounds like the Bikeshed of the user research world

As a startup / company, I would be more worried about actual referrals, customer churn/ retention, cost of acquisition, than NPS.

Low NPS maybe a sign of something wrong, but it’s likely also showing up in other survey questions. NPS may not be adding any value

NPS may be simple, but not necessarily useful

As a product manager, I become a skeptic when some startup or product touts high NPS scores with little else to back it up.

As an investor, I would ideally ignore the NPS score, or give it very less weightage unless backed by actual metrics. it is easy to manipulate and if it’s rewarded it would be in any company’s best interests to figure out how to get better scores.

Edit button on twitter is possibly the most requested feature in any product EVER, but somehow twitter never really introduced it. It is also possibly the only “Social Media” where you cannot edit as soon as you post. You can only delete and repost or add a correction in reply.

The feature is so highly requested that even twitter trolled the world by saying, you get an edit if everyone wears a mask. Great way to spread the message but basically saying, NAAAH

My Take  

Twitter will likely NEVER give an edit button. It’s not a simple social media. Tweets are public, they are quoted, and become news. Tweets are a system of records of a person’s opinion

Tweets are a system of records for a person’s opinion. 

There are no edits in it, only new versions. All history is maintained in perpetuity unless it’s useless. 
Each person is aware of exactly which version of persons opinion they interacted with.

Ideally they should not have delete but thats survival, you probably wont use it without an ability to delete.

I suspect if ever twitter decides to allow some kind of edit, it will come up with some kind of versioning system for tweets. 
Each interaction would be tagged to the exact version. 10 comments on version 1, 20 likes on version 2
But most likely that wont happen anytime in the near future.

While as a user I hate not being able to edit, I can clearly see its utility. This system of record is so powerful that you now have an authoritative view on what the person thought in the past. It’s either authoritative, or it has been deleted. There is no guessing. 

Its a phenomenal space to be in.

Twitter is more valuable because it does not have an edit

I see the value so much that If Twitter ever added an edit button like Linkedin or FB, I will probably sell whatever Twitter stocks I hold

What is a Fake door test

To put it simply, instead of building the feature your product simply pretends to have that feature and shows user a Call to Action to use that feature. When the user clicks on this CTA, you either tell the user that this feature is not yet ready and you have noted their interest, or tell them how they can use alternatives ways for completing their tasks.

Example: You are a Product Manager at twitter and twitter introduced a way to animate your tweets. This feature is only available on the app and you are trying to gauge user interest on desktop.

Instead of building this, you simply add an “animate this tweet” button on desktop and if a user clicks on it, you can tell them

” This feature is not currently available on desktop but you can always go too the app and use it, click here to see how”

How does this help

• You get data on potential size of the market

• You know which segment of user needs this feature

• You can validate almost all potential new features

Isn’t this disingenuous to the user?

It may seem so, but your job is to solve the most important needs of the user. If this helps you prioritise better, it is a win win for everyone. Sure some people will be a bit pissed but will be overall better served if you build the right features.

Some things to keep in mind

• Make sure the call to action is super duper clear. You do not want to be in a situation where users may get confused. Example: in Uber if we wanted to test if people wanted to schedule a ride on the web, giving users a CTA such as “Schedule a ride” would have been misleading because Schedule may mean, schedule for NOW. So a good CTA might be “Schedule for future”. You can work to fine tune the verbiage later

• Run this as an A/B if possible and only for a very small segment. You do not want your entire customer base pissed off

• When users click on the fake door make sure you explain it well. Don’t leave them hanging. Even better, if you do end up building this feature, maybe send a note to these users

• Measure effect on other interactions as well. You need to know the unintended consequences

• It is perfectly ok to not run a A/B if you don’t have the infra or enough traffic and collect data only for CTA interactions. But be clear about that miss

• Make it dirty and cheap. Don’t over think. Don’t end up doing user studies on what the fake door should look like. You are looking at directional results and your outcome should ideally be “Yes we are building this”, or, “No we are not”

Fake Door vs User Study and Surveys

User study is an awesome tool and can really help you understand the customer better. It also helps you test out multiple things at once.

Fake doors are super helpful when you have a very specific thing to know.
While you can also run surveys to gauge interest in certain features (Eg: survey to find if the new integration you are thinking of is valuable to the user), going by the old adage “don’t listen to what they are saying, but look at what they are doing”, Fake doors can be supremely helpful and surprisingly accurate even when not statistically significant.

Fake Doors for Validating Startup Ideas

This seems to be common. I have read multiple accounts, and have seen multiple examples, of people who setup simple websites with a real looking product page, and when you click on “Buy” or “Subscribe” they tell you its not yet ready but you can enter your email ID for early access.
If you gather enough email IDs, you have a winner idea and its time to Build

In product management, and In general one powerful weapon in your kitty is the ability to organise ideas and features in a logical structure. It helps you think better and in a more focussed way

These structures can be industry specific or domain specific. Here I present one:

My favourite structure, the one I use a lot, is the flow of information. It has 3 parts

1. Creation

2. Curation

3. Consumption

You can fit hundreds of things into these buckets

Eg : You are at Twitter and you want to improve Twitter

Think creation

• how to get more people to tweet

• how to get people to tweet more

• more ways to tweet

• what apart from tweets

Think curation

• hashtags

• lists

• personalisation

• follow

• organize in replies and threads

• curated bookmarked tweets

• trending

• add metadata to tweets -og tags

Think consumption

• how to view threads

• different devices and surfaces

• read me tweets

• api integrations to send tweets to external providers

• surface og tags,images etc

• tell me why am I seeing this

Sounds pretty simple, because it is. Breaking down into simple structures can help you focus your attention and prevent you from getting overwhelmed

So here goes the detail of stuff that helps me work better

PS: some links below are referral links and i make a small commission for qualified purchases

The Sitting Desk: This is perhaps one of my best investment yet. I got it made a few years ago.
My main specifications were:

• It should be longer than normal( I like to have books and reading material also on the table)

• Specific design(see the whites)

• No annoying leg support that most desks in the market had. They are a nuisance for slightly taller people

The standing desk setup: This is new. I figured the best way to have any standing time is if I take some meetings standing. Unfortunately most standing desks are either super expensive, or just not ergonomic. I also did not want to give up on my existing desk.

The key to a good standing desk is to have the monitor at your eye level and your arms at 90%, very similar to how it should be when you are sitting.

My standing desk comprises of :
Standing desk converter (I use this one) : I keep this always raised. It turned out to be much more sturdier than I thought and flexible height means I can adjust it just right.
– On Top of it is your stepper used in Gyms ( i once exercised 🙁 )
– a laptop stand( I use this one)

While this looks a lot, this gives me a comfortable standing posture. I can stand straight and type without straining my neck or my back

My Monitors
I primarily only work on my monitors and my laptop is only used when I am away from my desk or taking a standing meeting. I love keeping one monitor vertically as it helps me read long documents better, or open multiple windows together.
I use Samsung’s curved monitorfor my primary work. I love the curved monitor so much that I don’t think i would go back to flat ones any time soon. Also it seems curved monitors are better for your eyes.

I also use Amazon basics monitor armsthat can connect to my table without any drilling. I can shift it relatively easily to any side of the table. Monitor arms not only save space, they also allows me to put one of my monitors in a vertical position

Chair: I use a gaming chair by Greensoul. I assumed that gaming chairs should be super comfortable because gamers spend a lot of time sitting, but after using it for a year I think its not the case. It is a decent starter chair, but I am thinking of upgrading. Suggestions welcomed (Update: It seems they updated the lumbar support in the new model)

VC setup
My entire setup is fixed and I do not move around the laptop much, as a result I cannot use my laptop for my calls.
Also if you are using a monitor, and stare at it during a video call, it can be super distracting or annoying for the other person.
As a product manager, making at-least some personal connection is super critical and hence investing in a decent VC setup can go a long way (In budget)

Video: Logitech portable webcam this is attached to my primary monitor

Microphone: I use and a Ahuja MTP-20 wired unidirectional collar lavalier mike , which is attached directly to my Macbook. While the Logitech webcam comes with an inbuilt microphone, it’s voice capture is not that great. A cheap Lavalier microphone can work wonders. I use this unidirectional one that cuts out any background sound such as a running fan. Also, since it is wired, its pretty economical

Sound: Macbook Pro built in

A note on using bluetooth devices
I did experiment with a bluetooth setup for both microphone and speakers, but there was always this slight lag that made conversations unnatural. I recommend against using any bluetooth device for audio in or out.

Mesh Wifi: My life has been different pre and post mesh wifi. It has been the single best upgrade I have done to my home office setup. I use Deco M4.
See below the speed tests before and after mesh.
Directly from router Wifi: 22Mbps
On mesh: 160Mbps

Keyboard and Mice: Apple original Keyboard and Magic Mouse. If you are using a mac and can afford it (or your company pays for it), I recommend just closing your eyes and going with apple original. They not only work well, but also support all kinds of apple specific gestures, especially when it comes to the mouse

Other stuff

Wire organisers, lots of them to keep the wires always in place and mostly out of sight. I even created a charging station

Amazon basic lamp: Not only it looks cool, its great when you want to work in a yellow light during the night. Its rechargeable so no need to keep it plugged in. Really handy if you want to read a book on the desk.

Blackout Curtains: My desk faces the window, and while it is great to look out, during sunny days the brightness was way too much. My eyes started hurting trying to adjust to two different light sources.
I use these ones by Armenia Hague. Mine are fairly economical and work really well. I barely notice even if its super bright outside.( See images below)

Smart bulb :directly on top of my desk, connected to my google mini. Its fun to ask the assistant to switch off the light Tplink smart light

I do have another list on mailchimp, but I have been super irregular there as well. I will perhaps merge both lists at some point.

To kick this off, I thought it might be good idea to repost my recent blog post

Update: Post my article, Inc 42 also covered it and they were able to replicate my data. They also tested a few more apps with some very interetsing results.

Curious Case of Chingari app’s high Views and Likes

This was a difficult blog to write .

As some of you who follow me on twitter know, I am a fan of short form videos. Recently I decided to give short video apps popular in India a try. I tried Roposo , MX takatak, Moj, and Chingari.
After a few initial hiccups I was able to use all platforms.

I am bullish about the bharat / India story and believe we are primed for India first innovations

My plan was simple:
Upload the same video on all platforms and get a sense of onboarding process, the speed, community, India specific features etc. I did not expect any major traffic. I also set my language as Hindi because I wanted to experience the whole deal.

The video I used was me reciting Shiv Kumar Batalvi’s punjabi poetry . It had no music, was not following any trends, and was in punjabi. I had low expectations

I uploaded my video on all and waited. Every short video platform would typically show your video to some users and based on their interaction it might decide to show it to more people. Since the user base IMO is fairly similar across all apps, I was not surprised to see more or less same response on all. A few 100 views, and a few likes on all platforms except chingari. On Chingari I had 10,000+ views and 100s of likes, curiously 0 comments.

After the initial euphoria died down(hey maybe I went viral), 0 comments started to bother me. How are these views counted? how are these likes assigned?.

How views are counted is supercritical and can make or break a platform. It can cause real damage. Remember when facebook was overcounting views? It was a big scandal

To test my hypothesis that these views were not really earned by me, I uploaded a completely blank video. To my utter surprise, it followed the same trajectory. Similar number of likes and views after almost the same time.
I tried uploading more blank or nonsensical videos, just to see if somehow some video deviated from the pattern, but failed to find any. I literally had a video with a 5 second black screen. I found no way to not get views and likes.

To my annoyance, my empty videos had slight higher views and likes than my poetry. Am I net negative 🙂

Videos eventually settled at 250K views and ~17K likes. It was super uncanney. I also hit 1M views. Am I a genius influencer?

So I did an experiment. I created another account on a different phone and uploaded the similar video on both my primary account and the new account. Same title, no effects, no music, nothing and I took a screenshot every few hours for 2 days. I took a total of 8 readings and here are the results. The videos were literally a few second shots of my pillow and titled “bilkul kuch nahi” (Translation: absolutely nothing)

Video 1 direct Link

Video 2 direct Link

Just to make it clearer, lets plot the above values on a chart. The line chart was so close for both videos that it hid one of the lines, and I had to make a bar chart

Unfortunately I could not take readings at hourly basis, because work and life 🙂

Since views do seem to flatten around 250K, I am sure if I keep going it may not be such a straight line and there would be a decay.

I then plonked these values into a linear regression analyser (even though on a longer scale it may not be linear) and here are the results. It was not a perfect match, but was close.

Note: Chingari does not tell you exactly how many likes or views a video has, it rounds them off. Eg 2.5k instead of 2543 . This makes regression difficult

I was also curious to see how likes vs views were behaving. Looks like they were a perfect match

The formulae was

Likes on a video = 71.71084*Views in thousands + 3.34218

To see if this formula holds up, I looked at all the videos on my account and tried to predict the number of likes based on number of views. It was pretty close

Unless they change something, this seems like a replicable phenomenon. If you are a creator, I would like to know if you see the same numbers for dummy videos(Good videos would of course have engagement)

Why is this important

A lot of bharat users are using short videos as source of fame and may make a decision to pursue a career as an influencer based on traction they are getting.

They may also be using this to decide if they should spend more time to create or not. Knowing if their fame is real is an important question.
A sudden burst of views may also make people rate you better, as evident from this comment from playstore .

Views and engagement are also key metrics for any advertiser and media company. A very high number may encourage a company to invest heavily on that platform and ignore others. This happened in case of college humor which pivoted to facebook native because of insane engagement it was getting. The engagement numbers turned out to be overestimated

But Why?

Now I am NOT exactly sure why is this happening . Is this a new user boost? Perhaps a quirk of their targeting algorithm, maybe there are some users who like everything? Is this a normal trajectory for every video that has no engagement?

There are potentially alternate explanations and I would love to hear them.

What I can say is that it seems reaching millions of views on chingari app at the moment is a child’s play . Just upload almost anything and it might grow as per the formulas that I presented before. Even my own channel with useless videos has 1M+ views in a few days.

Do apps boost new users?

In the startup and product management world we call it a First run experience, and considerable effort is spent to make that as pleasant as possible.

Some companies do end up doing something special for the new user. This may include a boost in views by showing it to more people, but I have not seen this level of boosting.
This is also assuming that the numbers are all real. Actual people saw the videos and liked it.
In case numbers are not real, it is unethical and there is no excuse.

//update :
This does not seem a case for a New user. On my first account, which had 1M views, I uploaded 9 blank videos yesterday night. ALL of them have similar views and likes and the progression is still happening as predicted. Unless there is a different definition of a new user, this phenomenon does not seem to be limited to a new user

//end update

Things I did not check

When does the graph flatten : I presume it starts in day 3

Is there a trend in follow count

Quick Intro: I am Madhur Chadha. Product @Google. Ex @Uber (Built https://m.uber.com & other web ), @Microsoft (Office & Apps), Flock(Led Apps & integration), Youtuber, RJ, Poetry.

I write about everything from Product Management, to tech, to poetry.
Of all social networks, I am most active on TWITTER (follow me here)

My website

My Youtube

Subscribe now

In the meantime, tell your friends!