Tag: data

Musings

The breakdown continues

devayanitirthaliLeave a comment

The National Statistical Commission (NSC) is now defunct after the remaining two non-official/independent members (read: not political stooges) resigned on Monday. The reason cited was that the commission was not being taken seriously by the government. Their suggestions ignored; the reports they vetted were not published etc. There was also the controversy of tweaking the  economic growth data. The NSC was established in 2006 “with a mandate to evolve policies, priorities and standards in statistical matters.”

Massaging of data for political gains has lasting effects. It is impossible to gauge where we are and where we are going if we cannot depend on the consistancy of data collection instruments and methods of calculating key variables. Whoever comes to power next, the damage is already done and is irreversible. Now with NSC a non-functioning body there is no one to raise alarm or vett methods. In addition to all the fake news and emotional arguments, now we have data coming out of government agencies that is untrustworthy created by people with zero data ethics or academic scruple.

References:

  • Asit Ranjan Mishra. Two Members of National Statistical Commission Resign, Mint, January 30, 2019.
  • Govt sits on post-noteban jobs report, two top statistics panel members quit, NDTV??,  January 30, 2019.
Musings · Social Media, Technology & Education

Finding/creating Quality Data

devayanitirthaliLeave a comment

I have heard from Parag and many other people who work on India related topics about incomplete datasets, data parameters that do not match over the period making it impossible to look at anything over a longer time period. Just yesterday I was reading some of the DataCop blog posts about cleaning data as well as their apps that help collect data where data connectivity is difficult.

Today however I realized that one of the bigger problems of all is sanctity of data collected.
We received furniture delivery today. Overall, buying furniture and then choosing to buy it online has been a trying experience to say the least.  With this background this is what transpired:
The delivery guy updated the status on his mobile. Then he asked me to check the messgae I got. So far so good. Then the following conversation ensued:
Me: Ok received the message.
Deliver person: Click on it and click submit.
[Clicked on the link read the question ‘How likely are you to recommend Pepperfy to…’. The value was set to 10]
Me:  This is a feedback form. I will complete it later.
DP: Just submit it, it tells them how was our delivery.
[Stands next to me till I click submit]

How is that data point of any value to anybody?
I would have likely given an 8-9 or even 10 to that question and would have liked to give different values for different parameters. For example, 10 for professionalism of these particular delivery people but 3 for the overall process and communication. 8-9 to the studio personnel and the way they helped patiently. 7 for the website etc. However, that crucial data wasn’t captured. This problem is easily fixable by tweaking the feedback form.
The most problematic point however is the person standing there and making me submit the form because he perceives it to be his evaluation and linked to some kind of reward/punishment.

  • Is this perception based in truth? I hope not. Pinning the customer’s willingness to be a brand ambassador on one transaction out of many is ridiculous.
  • How did the delivery guy come to think of it that way? Was he briefed to nudge the customer to give feedback but something was lost in the communication and thought it was about him?
  • If somebody asked him to get it done in that particular way, what was their motivation? Are their incentives pushing them to make sure that they get 10s?

This particular problem is on one level about training everybody involved to follow data collection protocols. Followed by communicating why the data collection is important and what is done with it. Secondly, it is about making sure each person buys into the organizational vision and understands how the data helps fullfill that vision. If everybody is working towards the vision then collecting the data becomes a way to learn how it can be achieved rather than a check mark. Incentive structure (real or perceived) goes hand in hand with this. If the average rating is the basis for salary increases or commissions, there is a huge possibility that your data is useless.

In India I wonder if it is also about the basic attitude. Gaming the system and short termism is built in the DNA. More about that later.