As of now swr slave path disable only
when no usecase running, that is incorrect.
When stop the usecase itself disable the swr slave path and
set the clk for inactive bank.
Change-Id: I35bc693dc65028ba8ad04f59d47a1101c7bb470c
Signed-off-by: Yuhui Zhao <quic_yuhuzhao@quicinc.com>