The in-game limiter is virtually 0-lag. The Unreal Engine really has one of the best frame limiter implementations out there. So if you hit vsync, it should be noticeable immediately with the mouse.
The "bsmoothedframerate" thing just means the frame limiter is dynamic. For example, if you look one way in the game and your FPS drops to 50, then look the other and normally get 90, it just raises the limit slowly. It doesn't jump from 50 to 90 immediately, but raises it over a period of 1 second or so. This is a good thing, since "jump cuts" in frame rate are too jarring, even with g-sync.
It doesn't affect the frame limiter's latency. But if you don't like the effect (why?) then you can just set both min and max to same value. "bsmoothedframerate" must be "true" in all cases though, otherwise the frame limiter gets completely disabled, no matter what you set the min/max values to.
The "oneframethreadlag" option indeed lowers latency by one frame. This might interfere with multi-threading, which is needed on low-end systems to get acceptable frame rates. On a gaming PC with decent single-core performance, you can enable it. Didn't test it with this game. When I can't tell/feel any input lag in a game, that's where I stop tweaking it further
With all that being said, I had no issues with the frame limiter even before I got a g-sync monitor. I was using a 120.008Hz mode on my non-gsync monitor, and a 120FPS in-game limit gave zero lag. It was never hitting the 120.008 maximum. Same with RTSS. 120FPS cap on a custom 120.008Hz mode, and it gave low input lag (higher than the in-game limiter, of course, but much lower than uncapped vsync.)
So in the end, I can't really explain your results :-/ I suspect it might just be an indicator issue, and there's nothing to worry about. If you can't feel any input lag in Dishonored with an in-game 98FPS cap, then I'd say you're not actually hitting 100FPS, no matter what the monitor's OSD says.