Data-Mined Factors With No Theoretical Explanation Perform Just As Well

During their latest episode of the VALUE: After Hours Podcast, Taylor, Carlisle, Forehand, and Carbonneau discussed Data-Mined Factors With No Theoretical Explanation Perform Just As Well. Here’s an excerpt from the episode:

Tobias: Jack, do you want to walk us through your data mining paper?

Jack: Oh, sure. Yeah. It was actually very interesting. We had a couple of researchers on our podcast that’s coming out this Thursday. And so, we have this idea, like an investing, like an in factor investing, whether it’s value or whether it’s momentum. If a factor works in testing, and I want it to continue working going forward, I need some explanation as to why it works. And so, typically, what researchers will come up with is these risk base and the behavioral explanation, which is, the risk base is pretty straightforward. Value stocks typically are riskier, you’d expect. They have problems with their businesses, they’re cheap, they’re riskier than the market, I would want an excess return for that risk.

On the other side, the behavioral side would be people overestimate the problems with value companies. They beat down their stock prices. For people who are willing to buy those stocks, if they’ve overestimated the problems, that’s an opportunity. So, typically, those are the two explanations for any factor that we’ve used going historically to say, “Here’s why they should persist in the future.”

Well, we had a couple of researchers, Andrew Chen from the Federal Reserve and Alejandro Lopez-Lira from the University of Florida on our podcast this week. And the idea they came up with is they said, “All right, let’s test this. So, let’s take all the factors that have a risk-based explanation, let’s take the factors that have a behavioral explanation, and then let’s do a third group, and let’s just data mine the crap out of the accounting database.”

So, basically, let’s just divide everything by everything, let’s come up with the ratios that do the best, and then let’s use those on a standalone basis. And then, let’s take these three groups. We’ll do it the same through Fama-French. So, the testing period ends in the early 1990s. And then let’s see out of sample from the early 1990s forward how they work. And the answer is there’s zero difference between the ones that have the risk-based [Tobias laughs] explanation, the ones that have the behavioral explanation, and the ones that were just purely mind, which will challenge a lot of theory that a lot of us that are factor investors based what we do on if that ends up being true.

So, for instance, I asked him on the podcast. So one of the examples, I think was something like property, plant and equipment divided by cost of goods sold, something like that. Something you would never divide in the real world. But that had a similar return in sample and a similar return out of sample to something like momentum. And so, what they were saying is, basically, there’s really no reason you could say momentum is better than property, plant and equipment divided by cost of goods sold. So, it’s a really interesting thing just to say– And we were talking before we came on, talking about Robert Mercer at Renaissance, They’ve said all along that some of their factors that work best are the ones that have zero explanation or the ones that make no sense.

It’s just an interesting thing to think about going forward. We all rely on these explanations as to why these factors work. And if we test them and we don’t have an explanation, then we shouldn’t use them. But what if the ones that have no explanation perform just as well as the ones that do? I don’t know the answer to it. Certainly, academics that are smarter than me are testing this stuff, but I thought it was an interesting paper, and it’s an interesting conclusion.

Jake: Yeah, they’re almost polytheistic, “Lets just worship all of the Gods.” [chuckles] Maybe Toby’s a little bit more monotheistic of worshiping the value God [laughs]

Tobias: That’s right.

Jack: Yeah, even you can even worship it. I have no idea why this is working, which is a whole different change from whether you’re a momentum guy or a growth guy or a value guy, you usually have some basis for it. This is like, I’m dividing numbers. I have no reason to believe that dividing these numbers, other than the fact that there’s a lot of evidence that accounting data does impact stock prices. They did something else in the paper where they tested just mining tickers as opposed to accounting data, and they found no results there. They couldn’t get any good results out of just mining tickers. So, there is something about accounting data where it is meaningful. In terms of stock prices, the ratios we’re used to thinking about, they weren’t thinking about like, here’s what I should test, because I think it works. It was more like, just throw it all together, and whatever works, that persists just as well as the ones we could explain.

Jake: Is it possible though in a monkeys typing hamlet way that there’s just simply not enough data there to actually make that kind of claim. I know they’re looking at pretty large data sets, but if you’re going to just throw random numbers together, you can find things that will match over some period of time. But I would imagine like, you need just a big ass data set to actually feel good about betting on that going forward, don’t you think?

Jack: Yeah. No, I would think so. I think there’s definitely some randomness to that. But you can also argue– [crosstalk]

Jake: Yeah. Hit the noise element. There’s so much noise to filter that much noise out, you just need a huge sample size.

Jack: Yeah, exactly. And also, there’s a behavioral argument for– They’re not saying that the factors that have explanations don’t work out of sample. They’re just saying they work the same as the ones that don’t have explanations. So, it’s not really a challenge of factor investing doesn’t work. It’s really a challenge of, do we need these explanations for what we use? I don’t know the answer to it.

Jake: Let’s talk rationalization.

Jack: Exactly. But there’s arguments also like we asked them in the podcast like, do you– With the regular factors, did people mine the data to come up with book to market or come up with the explanation after the fact, or did they have the explanation first and then find book to market and the data? And they said they really didn’t know. Like, it depended on who did it, and so you could argue the other factors did it as well. So, it’s not something I really have a strong opinion on yet, but I just think it’s really interesting. The more I’ve been in the markets, the more I learned to challenge what I’ve learned and to say not to have these hard and fast beliefs and say, “No matter what I believe, you have to have an explanation for a factor.” I want to be open minded with this kind of stuff. So, I thought it was really interesting from that perspective.

Jake: I’ll have to ask Jim O’Shaughnessy next time I see him, what was he doing? Did he show up with the answers already and then try to back solve or was he following a more scientific approach?

Jack: I always thought it would be interesting to do like a– No one would ever buy it, but if you did like a factor ETF like X the ones that actually make sense.

Jake: Right.

Jack: So, you did an ETF of just the ones that don’t make any sense and used it as like a diversifying complement to your standard factor exposure, it would be interesting. No one would ever invest in it, but there’s so many– I mean, every ETF is taken these days, that’s at least one that would be– [crosstalk]

Tobias: I’m sure you can wrap narrative around. I’m sure you could say it’s like the– We’re just looking for signals where like– What’s that firm? Medallion? What’s the firm? Medallion? [unintelligible [00:24:31] Medallion?

Jake: Rentex.

Jack: Renaissance.

Tobias: Rentex. So, Renaissance, they’re just open about the fact that they don’t have any– There’s no explanation, we’re just going to test everything. But what you’d expect to find if you test tens of thousands of ratios through one data set, and then you find all of the ones that worked in that first data set, then you test them again through a second data set, there would be something that would survive just by pure chance, there are going to be things that survive through both data sets. But I think the more scary thing is what it says about momentum and probably value and other things too that you can’t even demonstrate that they are having survived two sets that even though there’s an explanation, like, they’re no better than the things that are cooked up by the computer.

Jack: What was interesting too is they all out of sample. It’s widely known that value out of samples had a lower premium than it did pre-1991. The ones that couldn’t be explained did as well. Not as much, but I can understand value going down. You could say, “All right, people became aware of it. They started following it. The premiums are less.” But cost of goods sold, divided by property, plant and equipment, those also deteriorated out of samples. So, we tried to understand that. I don’t really completely understand yet why that happened.

Tobias: It doesn’t make any sense at all.

You can find out more about the VALUE: After Hours Podcast here – VALUE: After Hours Podcast. You can also listen to the podcast on your favorite podcast platforms here:

Apple Podcasts