think that is not possible as you wished it.
On the left side of the challenge map you have a cog/wheel button.
You can switch on/off your found and owned challenges.
Every challenge with a write note has a small pencil symbol bottom right of the challenge. But you can't switch off the ones without this pencil.
Perhaps you can ask for this in the feature request area. Sounds like a good idea.
You have seen the tab within your profile statistics? For paid members you have a list of all your caches with a wirte note. All noted challenges which aren't logged or owned are listed.