In the fast-changing fintech field, the iteration of risk control models seems to have become the norm. However, this article makes a disruptive point: rapid model iteration may not be as effective as we think.
Strictly speaking, I would say that rapid iteration of the model is basically ineffective.
The applicable scenario is the risk control model, and I can't give you a clear definition of what is fast, but it must be in it on a weekly and monthly basis.
Let me argue this point of view, the purpose of which is to make you stop fooling around, and the other is to make you stop believing other people's nonsense.
My writing on risk control has always been philosophical and artistic, and I can't and don't want to write about technology. You don't have to trust the data too much, because the data has applicability, it is not necessarily conclusive, but you can trust me.
Let's start with the philosophical level
Argument 1: If credit is stable, then the credit model should also be stable
Credit is stable, this sentence can be directly used in front of the mathematical symbol of ∵, ∴ there is the so-called no trust, then the credit model should be very stable. However, the risk model has to iterate frequently, and there is only one explanation, that is, the modeling goal of the risk model is not equivalent to credit, but a representation of credit, which has a certain instability.
But in the end, this matter is relatively stable, so there is no need to iterate too frequently on the risk control model, like Souguangtui.
You have to have a degree. As for what exactly this degree is, of course, there is no conclusion. All I can say is that when you feel something is wrong, it's probably really useless, if you're confident in your abilities.
In fact, there are a few words that are important: the modeling goal is not the same as credit, but a representation of credit. It explains the uncertainty of risk performance, because when A is a representation of B, it means that A is an instance and not B itself, and there can be other representations. The form of risk is determined by users, platforms, market environment and other aspects.
Argument 2: Momentum strategies are not necessarily applicable to the credit risk control space
Under the fixed policy, if 10,000 users pass, is there a difference between using the first 8,000 samples and the last 8,000 samples? There is no difference, because there is no policy here.
Of course, if you insist on it, there is another factor that is time. Of course, this is negligible, okay, let's not ignore it either.
Time may make a difference between the two models, but it is not necessarily better or worse. It is slightly more likely that modeling with the last 8,000 samples is better, which is the belief that modeling with the most recent samples is better for future predictions. This is the momentum strategy, and the idea is that the trend always tends to continue. The same is true for the core idea of momentum optimization algorithms in neural networks.
As shown in the figure above, the effect of the momentum strategy will vary greatly between the sloping line and the volatility line. In fact, unless the swing line is sliced thin enough, it is not applicable.
Is the credit market environment a slash or a volatility line?
A bigger problem with momentum strategies is that there is a long performance period in the risk profile, such as MOB6 or even MOB12, which means that your sample is half a year or even a year away from now. Because of the existence of this gap, it is actually difficult to say whether the momentum is okay or not. A long gap may also be a cycle, right?
Argument 3: Model attenuation is long-term, mutation is short-term, and model iteration solves attenuation, not mutation.
Of course, the model will decay, but decay is a long-term word, and short-term change is not attenuation but abruptness.
What changes in the long term is that the customer base is constantly moving, and the risk relationship is also changing, which is attenuation.
The short-term change is that the effect suddenly decreases after the model is launched, this is because the model is used, and the lowest segment is gone, which is a mutation
Which is effective? Needless to say.
……
Okay, it's time to talk about why people think that rapid iteration of models is effective
In everyone's evaluation system, model iteration is useful because the KS of the validation set has been improving. But this is just your evaluation system, in real applications, is it useful?
Useless. Think about it the other way, if v2 is useful, then what about v3, v4, you will find that you don't have to do anything, you have been doing optimization, and you have improved a lot every time, why is the business not better?
You say you're attenuation, and you're quickly iterating to achieve the improvement of the previous attenuation diagram
What is the decay in a month? If there is, then the long-term installment product is all wrong. Do you say yes and no?
Whether the model iteration is real and effective, I wrote it before, and it was very well written, which is this article "On the question of which is better or worse for the old and new strategy models". Comparing the effect of the two versions of the model, what should be looked at is the customer group with a high passing rate, which is a high-quality customer group, and often everyone looks at the inferior customer group, because these customer groups have poor effect and are pain point customer groups, and only these customer groups have significantly improved their effect.
Friends, there's no point in iterating quickly if you're not doing anything.
So why do so many teams keep iterating? It's very simple, these people always have to find something to do, to find something to tell. It's just that no matter how much a system is tossed internally, it does not increase the knowledge and value outside the system.
"Celebrating More Than Years 2" in the spring season, Hou Ji often said that they came to the examination institute to move bricks and work not only to eat, but also to show their faces, but Guo Shangshu said that all the people who came would not be hired, because these people were opportunistic and had character problems. You see, sometimes performance can have the opposite effect.
If anyone says that they have a lot of iterations on their team's model, that they are very fast, and that they are proud of it, you can just listen to it, and you can even label the other person as not very reliable.
So how do you make it a little more stable? That way, you don't have to mess around. Of course, it is a high-quality user operation, identify quasi-risks, identify quasi-needs, evaluate revenues, evaluate debts, and don't mess around.
How did I start speaking from a regulatory perspective? It's kind of interesting hahahaha.
This article is written by Everyone is a Product Manager Author [Lei Shuai], WeChat public account: [Lei Shuai Fast and Slow], original / authorized Published in Everyone is a product manager, without permission, it is forbidden to reprint.
Image from Unsplash, based on the CC0 license.