How certain is that really? I read a very niche, but very well researched book about local violin makers in my region, and there were hundreds, and that "didn't include the ones who merely made one or two fiddles as a project" according to the authors. All the people they listed did it as a side gig to their farming.
People here had time they couldn't just sink into doing more farming and fishing. All primary industries have natural downtime. They very often spent that on specializing, often into useful things like cobbling, shipbuilding or carpentry, but also into less immediately useful things like making musical instruments or elaborate altar pieces for the churches. So what makes blacksmithing so special a guy who learned it would never "go back to a farm"?