I obsessed over my M-Sport and M3 wheel in my e92, it was life altering. I would wash my hands, without fail, before getting in my car every single time. Still after a few months the leather developed a slight shine to it. I had a few cleaners I would use that made the leather appear dull again, but the shine would always come back after a few drives.
I think the only way to keep it look powder dull like it is when its new, is to drive with a pair of knit gloves. I think even leather gloves would create enough friction to remove some of the matte clear coat applied to the leather.
Another technique I use is not sliding my hands on the wheel when turning. I would either grip the wheel firmly, or not at all-no letting the wheel slide underneath my hands ever. This started when I got an alcantara wheel, and after 3 years the wheel still looks immaculate.
So to sum it up:
•Always wash hands
•If needed, use knit gloves
•Never let the wheel slide underneath your hands