There's truth in both sides of it. Norway and Iceland do keep out because of the importance of fishing, but at the same time we've got to remember they are feeding far lower populations than the UK (5.5m between them), so aren't fishing as heavily as we are. The idea that Britain would've just allowed fishing to continue unabated is misguided, we'd have come under both internal and international pressure to impose conservation measures as fish levels depleted - even with the CFP its been a case of too little, too late for the North Sea.
The main crux of it is that Fishing, like Coalmining, was an industry doomed to shrink. The issue was the Governments did absolutely nothing to help the communities affected by this, leaving them to wither and suffer extreme hardships. With fishing towns they had a convenient scapegoat for their inactivities, which in a way they are reaping the side benefits of now.
I see it myself when I head up to where my family live in Morecambe, much like those on the east coast it was very dependant on its fishing industry (with a sideline in tourism), yet the town massively hit rock bottom in the late 70's/80s and is barely dragging itself back up - and this is in a very much Tory constituency (the rest of the constituency is very dominated by active farming). Even today the town centre has a sad feeling of neglect and being behind the times, despite it finally getting much needed investment - the M6 link which was offered to be built in the 60s by Heysham docks but rejected as 'unimportant' has finally been done at a massive cost, and that's had a huge difference as prior it was nigh on impossible to reach as it required travelling through Lancaster city centre and out on one gridlocked road. Even still, the local economy is dominated by Heysham port, and fishing is still it's identifying trade.
I can't say I've discussed at length the EU up there, but it does crop up in convo in the pub and at gatherings, and its a huge leave town. Fishing does get a mention, but still its immigration and 'sovereignty' that crop up. I say I don't discuss it much, because my mum is a massive leave supporter because of immigration, basing it on a few tales of woe from her ex-squaddie/prison warden husband than the 60 years of prosperity she had as the child of immigrants..... (trying to convince her that there is little difference between Windrush evictees and her own mother is the fact that being in the EU gives her right to stay is a tough gig)