IS250_6MT

I drive 07 IS250 with 6 speed manual transmission and I started feeling a notchy/bumpy section when releasing the clutch.
It makes it hard to control the clutch smoothly as my biting point is within this notchy/bumpy section and this increases the chance of moving my clutch pedal above the biting point too quick which either jerks the car or bogs the engine.
Has anyone else who drives the 6 speed manual IS250 experience a similar issue? If so, what have you guys done to fix this issue?

By the way, I have changed the manual transmission fluid last weekend and I started noticing this issue couple days after. I do not see how the transmission fluid change may be related to this issue but wanted to mention this just in case.
Any feedback would be appreciated. Thanks!

IS250_6MT

Additionally, the clutch/brake fluid level is full but the color of the fluid is bit dark so I am planning to flush the fluid to see if it solves the clutch pedal issue.
Since the clutch and brake share the same reservoir, should I flush both clutch and brake fluid?
Would it be not as effective if I just flush the clutch fluid?