Video Version :
Transcript
<Piyush Malik >The digital agenda is all about data analytics, cloud ai, ml, and emerging tech. That are transforming not only companies, but also in entire industries and even our lives.
Hi, my name is Piyush Malik. A “Curious Mind” who wears and juggles multiple hats as an engineer, Management consultant, practitioner, a builder, a thought leader, entrepreneur, and a C-suite executive.
I have been in the industry and navigate the value of data analytics, applied ai/ ml and emerging technologies. I have created strategies and transformation programs that helps them compete with the digital natives. We all need high quality data, data that we can trust to do our jobs, but the unfortunate reality is that most data is riddled with errors.
Bad data is a brutal killer. It adds time and cost to even the simplest of operations. Makes it more difficult to make good decisions. About 20% of revenue is wasted by a typical company cause of bad data. Further data is only growing more important, and it is growing ex whole, natural. You can't monetize bad data, and if you think artificial intelligence will cure your ills, wake up as the machine learning models trained on a bad data….the only yield fake results!
Today, let's meet Tom Redman, an author, a data provocutier, and a true thought leader. Many know him as the Data Doc. He shows people and companies how to attack bad data, how to focus the effort, how to measure, how to find it, how to eliminate root causes of errors, how to create good data and how to build quality throughout the value chain. He has published books and hundreds of articles on these topics in places like Howard Business Review and Sloan Management Review, and he teaches this stuff all over the world.
I had the good fortune of meeting him over two decades. when I served in the board of the industry body, I A I D Q, which is International Association for Information Data Quality which incidentally he had co-founded.
Let's go and meet Dr Tom Redman.
How’r you doing?
So, for the audience who doesn't know you Dr. Redmond, tell us a little bit about yourself. Who is Tom Redman
<Tom Redman> Well, thank you very much for asking that question and asking me to join you today. And please call me Tom, right? You know, usually Dr. Redmond means I'm in some kind of trouble and maybe we'll try to cause some here today.
But look, look, my background's pretty simple. I was trained as a statistician of the Bell Labs . During the time there, I was trying to figure out how to put more bits through the network. . It was a time at Bell Labs where if, if you saw a thread that you thought might lead somewhere important to telephony, you pulled on it.
And, and I saw several, but the one that led somewhere really was data quality. At the time, AT & T probably produced more data than any company in, the world. Google wasn't around at that time and, and Facebook and so forth. And and the specific thread I pulled on, had to do with the payment of access charges, which at that time were AT&T's biggest expense, over $20 billion a year.
And, there was a lot of concern that those charges weren't correct and , we were tasked with figuring out better ways to improve the quality of those charge. and we worked on that and, and got somewhere and, and as a result of that, set up a little lab, a data quality lab. And, I loved working AT&T . I left in, in 96 to set up my my company Data Quality Solutions.
And quite frankly, I think I have the best job in the world. I get to do two kinds of things. The, first thing I do is I work with companies all over the world and I try to help them understand their issues and opportunity, mostly in data quality, but, also in, in data science and modernization and so forth.
Set their strategies. So, I work with companies all over the world, and so people are really trying to make a difference in their companies. And then the other thing I do is, I'm sort of like the, you know, the number one data quality evangelist in the world. And, and and in the course of that, I've, I've built relationships with editors at Harvard Business Review and Sloan Management Review, and all of us in the data space owe them so much, so great a debt for getting stuff not just by myself, but by others like Tom Davenport and, and others into.
Into the eyes and hearts and minds of a, a broader business community.
<Piyush Malik> Well, that's wonderful to hear. And you know, the stuff that you are doing with Davenport and H B R, I've been a fan of that as well. And of course I've read your publications all these years. But, but you know, you're a lot younger than me.
But tell me. And when you got started to be called as the data doc and who were the first person who called you that?
<Tom Redman> Yep.
Well, well, thank you for that. I, bet I'm not younger than you, but let's not do that. You know, in, in, in public maybe sometime some other time, but so, so I, work with a number of clients and, and it and at some point, a is , working with a, particular customer.
They may not want to be named, but, but one of the, the guys there said, wow, you're like a data doctor, right? And he started calling me the data doc. He was kind of teasing me and, and so forth. And. , and I'm not sure he'd, he, he'd like to be named, but to, to be honest, you know, I mean, immediately I saw that, you know, that was pretty cool.
But I resisted putting it on my business card, . It was, you know, it just seemed a little a, little too silly or whatever it is. But once I did, people reacted so strongly to it, and, and I've, I've had that title for, I don't know. Closer to 25 years than 20. So, yeah.
<Piyush Malik> Yeah. I mean, I, I would always know you as a data doc when we first met at the I a DQ board meeting.
So anyways, thank you. Thank you for responding to that. So, your journey as an influential data leader and an author talk a little bit about, which was the first publication you published, and then how many have you kept a count of? How many have you published?
<Tom Redman> So , okay, well that, that's a really interesting question.
So, when I started in this work on, on data quality, on the billing example that I, that I talked about, you know, initially we thought this was gonna be a technical problem, right? You know, maybe there was something screwed up in databases or, or something like that. And and, and we found out, you know, there, look, there were indeed some technical aspects to it, but mostly data quality.
Was a management problem. So mostly it had to do with the way you thought about it, the roles and responsibilities you put in place. You know, quite frankly, what people fall on the trap of is thinking that data quality means we found and fixed all the errors kind of thing. And so we sort of pointed out that, boy, once you get into that game, you are in for a long, long haul.
And, and why not try to prevent them kind of thing. And, and so, You know, the technical aspects, they did things like set up control charts and, and that kind of thing. But the managerial topics were really the, the hard and, and interesting one. And and so I wrote my first, so now, by the way, remember I was trained as a statistician.
Mm-hmm. , I had, bill Labs, was not in publisher parish, but there was an attitude that if you came up with something, you know, something good, you ought to subject. to the withering scrutiny, you know, of your friends kind of thing, right? And, and so the first article I wrote was, was on that experience. I sent it off to Sloan Management Review and and doing that really changed my life.
A couple of weeks later, I got a call from a, a woman named Sarah Cliff and. and you know, I remember this story this way. She recounts, she doesn't think she'd ever be this, this unkind, but I recall her saying, you know, Tom, it's there's some really interesting ideas in this paper. It's a shame you can't write
Okay. But then what she did was she gave me little doses of things that I could do to become a better writer. And, and they're blindingly obvious. Like if the sentence has 12 words in it and, and you can rewrite it so it has eight, by all means do so. Okay. And, and she really got me thinking about, it's not just ideas, but it is selling those ideas.
And in data we, you know, we are selling these ideas and at least as long as you and I are alive, right. You know, we are going of them. Our most important job is selling these ideas to people who are unfamiliar with them, to people in PO position of authority to people who have data issues. and, and, and opportunities.
<Piyush Malik> Great, Tom, you remind me of the very first time I wrote a piece for i e e. And as you know, the, those papers are peer reviewed and then there is an editor, which is a sign, and I went to the similar exercise and I at the age of 30. When I first got that experience, it was like a blow. What? I don't know how to write
And he absolutely, you know, you know, the way that they do it. And by the way I have great regard for the editor who was assigned to me Cliff Pickover. He had, he had, he's an author of more than a hundred books and himself. A lot of patterns as well. So, and he happens to be an IBM er too. So I had, I had it took me six months to work with them to get to the final product.
And I said, well, in this time I could have written two books. So that
< Tom Redman> I, I learned so much from Sarah. And then quite frankly, I, I have, you know, I've had a sequence of editors like that particularly. At Harvard Business Review, it's, and for a long time I worked with a woman named Courtney Cashman more recently Tom Stackpole.
And, and not just are they, you know, not just are, they are sort of helping shape the ideas, but they're also really, you know, helping me just become a better writer, right? Say things quickly and simply and, and and, and. , you know, it's, it is interesting that the, the link to quality Right. You know, cuz quality's all about quality's all about meeting customer needs.
Yeah. And so, so, but repeatedly, well, who's gonna read this? What do you want them to take away? What do you want them to do? Right. Be very specific. What is the problem they'll be interested in? What should they do as a result? And, and, and by the way, I mean, that's pretty darn obvious. Yeah. Yeah. Yet, you know, you get in and you got all these ideas going on in your head and you're all excited and, and so forth, and it's easy to forget.
<Piyush Malik>Yeah. Okay. Yeah, yeah, yeah. Absolutely. So, so tell me, how was it how did you get introduced to Tom Devonport and how did you start collaborating with him?
< Tom Redman>So I, I'm, I'm not sure I know the answer to that. I mean, obviously, you know, I've read a lot of his stuff and I I don't know. I, I think I probably sent him an email or something and, and, and said, you know, Hey, I like your stuff and you know, would you like to would, would you like to meet?
And, and he was in New York. I, I live on the Jersey Shore. He, at the time lived in Boston, but you know, he agreed that next time he was in New York, I tried to come in and we'd meet. face-to-face and, and you know, and what we found is, is that, you know, we, we we, we looked at things from, from different perspectives and it's became, you know, very, very fruitful.
I mean, I'm a consultant, right? You know, a advice. I mean, he is a researcher. And, and, and, and very, very disciplined. By the way, one thing that's really important I'm trained as a statistician, right? I, I, he's a social scientist and I don't remember whether he was an anthropologist or maybe a sociologist or, or whatever, but you know, like we keep talking about all, all the real problems.
well, the social issues, the politics and, and so forth. And, and so, you know, so that overlap has, has proven really, really fruitful. We've never worked very hard on a paper together. It's just, you know, we've kind of, eh, you know, you interested in it? Absolutely. Yeah. I am. And, and then boom, you know, three or four weeks later, something, you know, pretty good is in the works.
< Piyush Malik>That, that's great. You know this is, this is the takeaway for the youngsters who are watching. Never be afraid to ask anyone for help or for collaboration.
< Tom Redman>So, you know, I'm so glad you brought that up. I, throughout my career, right, I've reached out to a lot of people and not all of them have said yes, but when, when I, when I was young, I mean, I did that to people were incredibly generous.
Right. You know, some work with Tom Davenport. Now I work with John Laley. Right. You know, some work with Doug Laney. Right. Roger Hurl. Right. You know, people all over the world and it just enriches things so much. But the point is, I never, I always take the call. Right. You know, you have an idea you want to talk about, right?
I mean, I may not make a second one, right? , but, but I mean, and I'm really interested and, and what you just said, I want to double that, right. Reach out, right. If you're in doubt about reaching out to me, reach out to me. Right? Info at Data quality solutions or linked. Or whatever it is. I want to hear what you're up to.
I want to encourage you and you know, and if, and if you got great ideas, if I can, I want to help you.
< Piyush Malik>This is the sign of a true leader. Guys, I have to tell you a story here. When I first met Tom, it was when I was leading the Pricewaterhouse Coopers Data Integrity Practice, which is related to data quality, and it dovetailed into what I A I D Q was doing.
And so, so we had great discussions in industry conferences and all that. And then there was a time that I was. at a D G I Q conference and Tom brings up "So Piyush, what do you think about the thick paper that McKinsey released?" And that paper was on big data and so you can imagine what timeframe it was.
And I had recently been tapped to lead the big data analytics center of excellence at I B M, and I told, "Tom, very timely because that paper is sitting on my desk. And right after this conference, I'm gonna go back and read that paper" and then we talked after that. So in, in this is, this is, this is something to say that, you know there are angels....
and there are experts out there, and you should, you should always reach out. And they are great folks to collaborate with in the rest. Some of them Tom has already mentioned. And I'm sure we will meet more of such folks as we continue our conversations in this data leader space. So I would pivot now in the last few minutes that we have to.
Exactly are you working on currently and where do you see the world of data management go in the next five years or so?
< Tom Redman>Yeah, I'm really glad you asked that question. The last I'm gonna say two years, 18 months, whatever. Have been as productive in my career as anything. And, and in some respects, maybe a bunch of things just came together during the pandemic.
I had a, you know, some study groups coming and, and, but the first thing that, that, that I really, you know, hit on, and in retrospect it is blindingly obvious. But I use the term regular people to be those who do not have data in their. Right, and what became immediately clear was we can't do anything in data without regular people.
Right? You can't do a data science project. Without understanding the problem. You can't do a data science project without understanding the data. You can't do a data science project without understanding the context. And if you're lucky enough to come up with something you can't do a a, you can't complete the project without involving them in the solution.
Okay. In data qualities, right. We, we, we, people create data. People are customers of data. We can't improve data quality. Without data creators and data customers stepping into those, in, into those roles, and, and so this is, this is a big, big deal. The most important people in data programs are not data scientists.
They're not chief data officers. They are regular people. . Right? And they are regular people. And, and if we're in the data space, we have to figure out how to engage them. And, and by the way, if you're a regular person, there has never been more opportunity to get yourself involved, right? You, you, you can do things like improved data.
In your shop, you can lead small data quality improvement projects. You can reach out to data scientists who may be sitting in the lab right, in their, in their centers of excellence or labs and, and, and not knowing what to do. You can be the game changer, right? And, and so I'm both excited. I'm excited for regular people cuz they can do this.
We've, we're, we're, we're using the term, the data generation for regular people who step in and, and, and put themselves in, in, in the data generation. I hope, by the way, you know, being a member of the data generation will, will pretty darn soon. Be as cool as being a member of the Pepsi generation, right?
I mean, you know, young and hip. And, and this is the way th th this is the way of, of the world. And, and so my work has been recognizing that, calling out the data generation and then exploring the implications. What does this mean for management? What is, what is the organization for data look like? Right?
How is it that a small chief data office is gonna create leverage with, you know, the, the. The many more people who don't have data in their title. Okay. And and my clients that are on the leading edge have managed to do this. And, and you know, of my most recent articles have have focused on various aspects of this.
< Piyush Malik>Great, great. Now you are making a difference. You have been making a difference all these decades that I've known you and I, I, I tend to call you a data integrator. Basically you are waking the management up and now you are waking the even young steps up because, you know, the youngsters get enamored by cool technology.
Big data is hard. AI ML is hard, but you. None of those models work if the basic data foundation is not there. If the basic data quality and hygiene principles are not adhered to, and that's a message that you've been giving all these years. You and I have been talking all these years, but I think now is the time to reengage and, and I don't want to use the phrase making data great again, but again the important thing.
We have to establish the value of data everyday. Common man everyday person in, in the organization that you mentioned the creating the data generation. So thank you very much Tom, for doing all that you do. And in my tradition, this is namaste from my soul to yours thank you for the time and let's continue our collaboration going forward.
< Tom Redman>Thank you. I very much look forward to that. Thank you so much for reaching out and good, good luck to you. Good luck to all of us and, and again, please join the data generation.
< Piyush Malik>Absolutely. Thank you, Tom.
The Digital Agenda: Data Quality Management