The 2 N3 my son and daughter own were bought brand new and there was no problem with them. My N3 was bought used and luckily I had the store warranty (factory warranty is non-transferrable), but I had to have a Yamaha tech come to my house twice to repair it under warranty. The first issue was that a couple of octaves in the lows did not sound loud enough even when struck harder. The second issue was that a few octaves in the mid areas would not trigger a sound consistently when struck very lightly, a hit or miss situation despite the same soft strike to the note.
Anyway, the tech had to replace a couple of sensor boards (the hammer sensor board and the velocity sensor board), then recalibrate all the keys thoroughly before everything works again. So what I'm trying to say here is that if moving it around for different acoustical conditions still doesn't help, it's possible that either the keys in that area need regulation, or it's also possible that the sensor boards in that area have become flaky for some reason. It may be the sensor board, it may be something out of calibration, it may be the velocity gradient films getting dirty in that area, or even a combination of things... I don't think it's likely an issue with the sound engine. Like you said, if it's so, then everybody would have complained already. It's more likely an issue with the action/sensors, causing the sound engine to sound bad.
At least if it sounds bad just the same through the headphones, then you've eliminated the amplification system as the culprit.