Will over drying problematic or oily skin clear up acne?
The opposite is true. When skin becomes overly dry, an environment is created in which the skin is signaled to produce more oil.
Use drying products (such as benzoyl peroxide) sparingly, and keep oily skin hydrated with non-comedogenic moisturizer where needed.
The goal is to keep skin balanced.