TDA EP6 Video
TDA EP6 Summary
The conversation between the host, Piyush Malik, and Dr. Tom Redman, who recently published a book titled "People and Data: Uniting to Transform your business”. The discussion centers around the importance of data, its governance, and its impact on individuals, organizations, and society
Dr. Redman explains that data is essential in daily life, much like air and water, and it plays a crucial role in decision-making, problem-solving, and innovation. He emphasizes that Data Goverance covers a wide spectrum of things but data quality is vital, especially in the context of AI and generative AI, where accurate and clean data is essential for meaningful results.
The conversation delves into the significance of trust in data and its broader implications in various realms, including politics and business. Trust is seen as the "coin of the realm," and without it, progress and good outcomes are challenging to achieve.
Dr. Redman also emphasizes the role of regular people (non-data professionals) in managing data and the need for proper training and support in data management tasks. He believes that empowering individuals to engage with data will lead to more significant contributions to the field and enable them to make a positive impact in their organizations.
Key Points:
Data is essential in daily life, similar to air and water, and it plays a critical role in decision-making and problem-solving.
Trust is fundamental in data, and without it, progress and good outcomes are difficult to achieve in various realms, including politics and business.
The “DataGeneration” comprises of regular people. And these regular people (non-data professionals) are responsible for much of the data management, and they need proper training and support in this area.
AI and generative AI require accurate and clean data for meaningful results, making data quality & its governance paramount.
Empowering individuals to engage with data will lead to significant contributions to the field and positive impacts in organizations.
Full Transcript
Piyush Malik: [00:00:00] Digital Agenda is all about data, analytics, cloud, AIML, and emerging technologies that are transforming not only companies, but also entire industries and even our lives. Hi, my name is Piyush Malik, a curious mind who wears and juggles multiple hats as an engineer, a management consultant, practitioner, a builder, a thought leader, entrepreneur, and a C suite executive.
I have been in the industry helping organizations realize and navigate the value of data, analytics, applied AIML, and emerging technologies. I [00:01:00] have created strategies and transformation programs that helps them compete with the digital natives. We all need high quality data. Data that we can trust to do our jobs.
But the unfortunate reality is that most data is riddled with errors. Bad data is a brutal killer. It adds time and cost to even the simplest of operations. makes it more difficult to make good decisions. About 20% of revenue is wasted by a typical company because of bad data. Further, data is only growing more important and it is growing exponentially.
You can't monetize bad data and if you think artificial intelligence will cure your ills, wake up. As the machine learning models trained on a bad data set.[00:02:00]
Today, let's meet Tom Redman, an author, a data provocateur, and a true thought leader. Many know him as the Data Doc. He shows people and companies how to attack bad data, how to focus the effort, how to measure, how to find it, how to eliminate root causes of errors, how to create good data and how to build quality throughout the value chain. He has published books and hundreds of articles on these topics in places like Howard Business Review and Sloan Management and he teaches this stuff all over the world. I had the good fortune of meeting him over two decades ago when I served on the board of the industry body IAIDQ, which is International Association for Information and Data Quality which incidentally he had co founded.
Let's go and meet Tom.
Hello! The Digital Agenda today is delighted to have Dr. Tom Redman back and first of all, congratulations to Datadoc on publication of your latest book, People and Data. Transformed your business. I just got it and I am waiting for my downtime this week to get through this book, but I know it will be an interesting read.
I already started reading through some pages, but let's, let's welcome Dr. Edmund and Tom, tell me how was the process of writing this book. And this is this is a great contribution to the field of data. Well, thank
Dr Tom Redman: you very much for that for that question. By the way, if I understand [00:04:00] correctly, let me see if I have this correct.
Weren't you just named a Top Gun on LinkedIn? Or did I misread that and you're a top data governance voice, which was?
Piyush Malik: Well, well, well, you know & thank you for observing that “ data doc”, you've been going with the Moniker “THE data doc”, but, you know, of late, LinkedIn has introduced something called collaborative articles and I've been contributing to a lot of those.
So perhaps that as well as the engagement that I'm doing on social media with leaders like yourself , And I think it's not getting unnoticed. So , I'm thankful to the readers as well as to the LinkedIn platform to have seen that. So again, it's all in the service of the data community that both you and I are.
And of course, you've been a great mentor to me in the past.
Dr Tom Redman: Yeah, well, thank you very much. And I'm so happy you're joining the effort because I'm more and more convinced we're not going anywhere in the data space until we get large numbers of people and [00:05:00] hopefully everyone engaged and you know, it's, it's been tough going so far.
So look, the primary, I made 2 observations in the process that led up to people and data. And, and the first observation was, was really, really very simple, and that is almost all actual data management is done by regular people. Regular people are those who don't have data in their title rather than data professionals.
And, and, you know, they may be done by somebody in marketing or sales or operations, finance, whatever it is, But they just need data to get through the day. And the stuff they're presented is obviously aired. And, and so they have to do the work , of spotting those errors and then figuring out what to do with them.
It may be correcting them. It may be some things look [00:06:00] suspicious and they have to seek confirmatory evidence, but but practically everybody is doing this. Practically. Everybody is spending a significant fraction of their time doing data management tasks, and they're doing it without any training. Any guidance, any support, any, any real tools.
And I just want you to think about that. And so data is an increasing fraction of everyone's job and not having this training. It's just. It's absurd as it's as absurd as, as a barista at a Starbucks, not knowing how to make a cup of coffee, right? You just can't imagine that. And, and so that was you know, that, that, that was a big motivation.
And then the 2nd motivation was simply the observation. You look at. Any of our, the process of which we do any of our work, like, like a data science project [00:07:00] or a data quality project or an architecture, whatever it is, and the work can't be done without regular people, right? So a data science project starts with define the problem.
Well, no data scientists sitting on their own can properly define the business problem and, and the context. And And so these sort of were, you know, the motivators that led to the work that that, that led to people in data. One other thing, I mean, and this is kind of, I'm, I'm really proud of this. What was the observation?
That when people learn to attack data quality problems, or they learn how to use a little bit more data in their day job, how empowered they become, right? They empower themselves and. And and in some respects it's as, as mundane as, well, I don't have to come in and correct as many errors today so I can do my sales [00:08:00] job.
But it's more than that. I got to the bottom of something. I fixed something. These things aren't coming back. And, and in some respects, I find that, you know, people have switched on to data. They're, you know, they're as insufferable as a six year old is when, when they learn to ride a bike. And, you know, you remember this, they can't ride, they can't ride, they can't ride.
All of a sudden they figured out their, you know, looked at, you know, and then, you know, 20 minutes later, looked at one hand and, and stuff like that. Freedom of doing that, right? Learning to ride the bike and the freedom that comes from not having to deal with, with other people's junk in the data space is as empowering as that.
Okay. And, and so those were the, you know, the, the really primary lessons that this is very, very different than, the typical data.
Piyush Malik: Wonderful, Thank you for sharing that. And you know, As you [00:09:00] were talking about the analogy of a kid learning to bike, you know, it makes me think perhaps here is The data doc coming to free us from the shackles of the data drudgery.
Because nobody in an organization wants to enter data or clean data, but they want results. They want insights. And especially in the era of AI and generative AI, you know, it's all got to be trained on pristine data. And if we don't understand. What it can do for an organization or for individuals in organizations as well as in life, you know, then we are missing something.
Dr Tom Redman: And I want to bring you back for a moment, I mean, let me just build on this point for a second because we're generative AI. I mean, all the hype and stuff like that, right? It's not going anywhere without quality data and you're not getting large amounts of quality data without [00:10:00] large numbers of regular people, right?
I mean, you know, most people do when they hear this, they, they shake their head up and down just as you are. This is self evident, true, right? But unfortunately, some of the most companies, right? Yeah. You know, if they've got 100 to spend on this, they're spending, you know, 98. 50 on their algorithms and a quarter on on the data and a quarter on people.
And it's just not enough. This has to level out. So, if all you care about is generative AI, right? Well, the 1st thing you better worry about is the data. And the 2nd thing is the people and you ought to start that today.
Piyush Malik: Absolutely. And this Generative AI is a term that hasn't left me since last November.
Every conversation I get on with my clients or my peers or even my protegees in the industry, everybody's concerned about generative AI and rightfully so, you know, the pace of development in this AI Space is so fast, [00:11:00] newer companies being launched, newer opportunities being presented, but you're absolutely right without the right data, they can't go any further, you know, you’ve perhaps seen the cases of chatGPT hallucinating, that's nothing but the model not having the accurate data. during training. And these days, both open AI or Microsoft, as well as Google Bard, they say, we cannot rely on the accuracy of this data coming out of this generative AI. Yeah. Why is that? Because we haven't cleaned the mess of all those past years.
So now what is happening is people are coming up with enterprise specific generative models.Which takes some component from these open source models and then their own private models. Which are trained on company data on solving customer support problems and the low [00:12:00] hanging fruit of increasing productivity. And I think there are great progress being made in that space. But, you know, I don't want to digress and move away from the real purpose while we are here today.
I really want to get to certain things from this book. Then with your permission, I would like to read and get your opinion on.
Dr Tom Redman: Yeah, please, please do. And by the way, I mean, like, you know, we've been saying in the, you know, long and we've heard garbage in garbage out for our entire adult lives. Right?
And, you know, so it's still, it's still true. There's lots of opportunity to do good with AI. But the ability to do increasing amounts of damage or is there as well and, and bad data will be a prime contributor to that.
Piyush Malik: Absolutely! And as we talked about in the the digital agenda episode, number one, that was the theme that you and I talked about.
But today in your chapter one, you, where you [00:13:00] talk about and data Usually, most people don't give data a second thought, but much like air and water. It is essential to daily life. What a beautiful way to start.
Dr Tom Redman: Yeah. Yeah. Thank you very much for that. I mean, look, look, you know, if you're going to talk about people and data, it's, it's like, well, what's going on, you know, with people.
And, and so to start this book, I mean, this, by the way, I worked with Kogan Page on, on, on this book and Kogan page was absolutely terrific. And, and, you know, we had a discussion about this. They really helped shape this. Thank you. This first chapter, but, but we really try to take a regular person, right? Just, you know, a regular woman with a regular job and marketing and, and, you know, and a spouse and 2 kids and, and, and, and, and the things that she's going through in her, in her daily, you know, daily life and, you know, some of it is as mundane as, you know, As you know, [00:14:00] well, which way do her kids like their, you know, their peanut butter sandwiches cut, right?
Because they're going to go spend the weekend with their grandparents to write some, you know, some trouble she has at work and, and, you know, thinking about whether they can afford a house. Their kids are getting older, thinking about moving from the city to the suburbs and and so forth. And what was interesting is, is I wrote that.
I mean, I tried to make this just as basic as I could, but. But how many times she ran into data and, and all kinds of, of different kinds of data just during the course of her day. And, and of course, you know, we talked about the excitement around AI and, and the kinds of things it will do for a company.
But part of the excitement, you know, I think the bigger, the exciting, the thing I'm more significant about is just making her day a little easier and a little better, right? Making her kids happier, getting them educated, [00:15:00] maybe saving her some money, maybe making her job more satisfying. I mean, that to me is the spirit of people in data.
Yeah,
Piyush Malik: Absolutely. And through this book in some way, I also read that you have six kids and grandkids as well. Have any of your... I have seven grandkids.
Dr Tom Redman: Seven grandkids. Yeah. I count my kids and my in law kids as six. So, right. Yeah. Okay.
Piyush Malik: All right…Wonderful. So, just curious if any of them have read your book?
Or you intend to make them read it someday, during the Christmas period!
Dr Tom Redman: . Yeah, so, 1 of the things that's really interesting about that is, 1st of all, on my wife has insisted that every word I write for the last 15 years, she can understand and, and she's really demanding and and so. I also have a business manager who like, you know, makes my, [00:16:00] my wife look, you know, easy to, to work with, but, but a lot of the ideas and people on data, they got field tested in raising kids, right?
They get, they got field testing and raising kids and how negotiations with them would go and, and so forth. And, and so look, I mean I think they'd all consider themselves regular people and, and they use 50 times as much data in their days as I do now. And, and you know, so so, since I've given them their copies, I've heard, you know, from each of them.
Oh, that's really pretty good. Dad. You know. Kind of thing. And imagine how hard it is to get one of your kids to go. That's pretty good, dad.
Piyush Malik: Yeah, absolutely. And now, you know, this is, this is, this commands respect. You are contributing to the vast body of knowledge in the data and data and quality and data governance space for so long.
And to still have admirers at home, I think that's commendable. And, and I know you, you [00:17:00] attend conferences as well. I mean, I remember our conversations during the DGIQ conference where We would sit at the breakfast table talking about a topic and then you never know that conversations would go from five minute conversation to a hour long conversation and we'll keep debating pros and cons of that topic.
Do you think today's generation with the, what I call as the ADHD infused kids, the, they have the attention span of less than a minute in the age of TikTok. Yeah. Would this sit down to read a book fully from end to end?
Dr Tom Redman: Well, so, so look, I mean, I, know a lot of people are saying, you know, that's gonna be a real problem.
And, you know, they're making fun of the, of, of, of the next generation and so forth. I, I'm pretty sure our fathers did that with us too, you know, kind of thing. Yeah. And we turned out, all right, so I, I, I just look at my kids and, and, and the cohort of, of, of that, [00:18:00] you know, they're cohort, right? They're friends that I know.
And the fact of the matter is, they're just terrific human beings. Yes, they're trying to do the right thing in the world. They're, you know, they're, they're trying to figure out how, how, how they fit in. They're a lot less tolerant of, of, you know, malarkey at work kind of thing. Right. But they're, they're, you know, they're very much more social.
And, and, and so when you write a book now, I mean, You know, you know, if you look at those chapters, I mean, none of those chapters are very long. And I
Piyush Malik: noticed that. Yes, absolutely. Right.
Dr Tom Redman: So, you know, so, okay, well, you got, you know, 10 minutes on the train, then the, then you can read a chapter. The other thing is, is, I mean, again, Kogan page and, and the editing, I work really, really hard and work with editors to make this an engaging read.
Piyush Malik: Absolutely. From whatever I have read so far, it's very engaging. I want to read another small snippet from, with your permission, from another chapter here, [00:19:00] where it says all roads lead through quality. And when the topic is when trust leaves the room, George Shultz, U. S. Secretary of State under Bill Clinton on the importance of trust famously remarked then, trust is the coin of the realm.
When trust was in the room, whatever room that was, The family room, the school room, the government room, or the military room, things happened. When trust was not in the room, good things did not happen. Now, this statement is very, very relevant today in both politics and in the world. So care to elaborate a little bit more here.
Dr Tom Redman: Yeah, look, I mean, I, I'm not the 1st one to remark on this. I mean, it's obviously George Schultz is making a really, really important observation. And as you say, at every level, and then that McGilvery's talked about it in the context of quality. Right? I, [00:20:00] I, it's just, it is just so, so insightful.
And also, you know, sort of abuts the conversation we were having earlier about, you know, the younger generation and, you know, and building the kinds of relationships that that you need to to to create trust and and so. Right. I mean, I think we're, we're at this, you know, perhaps I hope we're at this inflection point in human history where we can absolutely start leveraging data more and more.
And and if we can, I mean, I think we can create an economic miracle, right? Like industrialization did, but that will not happen without. Without without trust and trust at all levels. And for me, you know, the all roads leads through quality in the political realm, right? Says, at least try to tell a complete and accurate stories that we, [00:21:00] you know, we realize you're aligned with one party or one perspective or, or, or something like that.
At least try. To tell a complete and an accurate story and, and, and and work with others to do so. I mean, I spend most of my life in advising companies. And the reason we have such data quality problems is the person in marketing is generating leads data and sending it on over to the salesperson and the salesperson finds all kinds of errors in it.
And then they kind of swear under their breath and go correct. The data is, you know, and, and, but well, sometimes I just like, well, why don't you call him up? Why don't you offer to buy him a pizza and let's discuss those errors. Let's, you know, figure out what's going on. Maybe we need to put some measurements in place or whatever, but, but rather than just making it, oh, those guys over there.
It's at least put the effort in [00:22:00] putting a face on. Yeah, absolutely. And I, this is another thing that I'm so excited about. I don't care who you are. I don't care what your level of education is. I don't care how afraid you are of data or anything like that technology. I don't care if you can work a spreadsheet.
You can do something in the data space. Now, you can do something to change the attitude, you know, if, if, if nothing else, you can say, I wonder who uses the stuff I create. Thank you. Let me go, let me go talk to them. And, and you know, I mean, I don't want to get, you know, you know, too much this way, but I mean, that to me is, is the really hopeful message here, right?
Is we don't, you know, obviously, I hope leadership gets it. I hope CDOs place more emphasis on this. I hope department heads do. I hope people who have business problem. I hope political leaders.[00:23:00] I hope political leaders take it upon themselves to, you know, think a little bit more broadly. And as I say, try to develop a complete and accurate picture.
But whether they do or don't, you, whoever is listening, you can do something and you can start it today and you can start it in a constructive, in a constructive way. And once again, I mean, I think that message is just so hopeful.
Piyush Malik: Absolutely!! Leaving us with a positive message about the power of data, about how a common man, common citizen, whether in life, whether in government, whether in organization, they can all contribute to it.
And we need to have a sense of ownership. To have a semblance of control over our destiny . About 10 years back when I was doing a keynote presentation at a conference. You know, in those days of big data and I was leading the big data analytics team at [00:24:00] IBM at that time. And I stood up on the stage audience of about 500 people in the conference room.
And I said, The next world war is going to be over data and we are already seeing signs of that. And I wish peace on everyone. And I wish that we all develop a sense of camaraderie and friendship with the concept of data and, and The good work that you have started with the data manifesto at one point, and then your articles in Howard Business Review and MIT Sloan Review, and in the past through your other books and body of work and consulting work, Tom, I'm very, very appreciative of what all you have done, and congratulations again to you on the publication of your latest book. I wish you luck and hope everybody goes and reads this and we [00:25:00] will reconnect pretty soon. So with that, thank you so much and namaste. Namaste.
The Digital Agenda: Data Governance: People And Data