Fruit absolutely matters (Matthew 7:15-20).
Since you think so, explain the parable of the barren fig tree.
You don't think it matters?
The Parable of the Barren Fig Tree
6And he told this parable: “A man had a fig tree planted in his vineyard, and he came seeking fruit on it and found none. 7And he said to the vinedresser, ‘Look, for three years now I have come seeking fruit on this fig tree, and I find none. Cut it down. Why should it use up the ground?’ 8And he answered him, ‘Sir, let it alone this year also, until I dig around it and put on manure. 9Then if it should bear fruit next year, well and good; but if not, you can cut it down.’”
Here's a full grown fig tree that had not born fruit for 3 years, & was going to be cut down if nothing in another year.
If it wasn't full grown, the owner would have expected nothing from a sapling, now would he?
You are missing the point of the parable of the fig tree. Jesus told this parable after reminding His listeners of a tower over the pool of Siloam which unexpectedly fell and people were killed. In Luke 13:3, we see the moral of the story is,
"unless you repent, you will likewise perish." To reiterate this moral, Jesus tells the story of the fig tree.
The fig tree has clear symbolic significance of the nation of Israel. The vineyard owner expresses his disappointment at the fruitless tree. He has looked for fruit from this tree for three years, but has found none. This three year time period is significant because for three years John the Baptist and Jesus had been preaching the message of repentance throughout Israel, but the fruits of repentance were not found from Israel as a whole.
John the Baptist warned the people about the Messiah coming and told them to
bring forth/bear fruits worthy of repentance because the ax was already laid at the root of the tree (Luke 3:8-9). But the nation of Israel had rejected their Messiah and had Him crucified.
In the parable, the vineyard owner grants another year to the tree. In the same way, God in His mercy may grant us another day, month, year, but if there is no fruit of repentance, His patience will eventually reach an end, and the fruitless, unrepentant individual will be cut down.