audio-lnx: soc: pinctrl-lpi: fix rmmod issues

Deregister lpi with audio notifier when removing driver to
avoid crash when rmmod and insmod again.

Change-Id: Ibcaf272e6405ef8403b3ac66af5572050fa1092e
Signed-off-by: Meng Wang <[email protected]>

soc/pinctrl-lpi.c

@@ -622,6 +622,7 @@ static int lpi_pinctrl_remove(struct platform_device *pdev)
 {
 	struct lpi_gpio_state *state = platform_get_drvdata(pdev);
 
+	audio_notifier_deregister("lpi_tlmm");
 	gpiochip_remove(&state->chip);
 	return 0;
 }