The disclosed solution includes a method for dynamically switching modalities based upon inferred conditions in a dialog session involving a speech application. The method establishes a dialog session between a user and the speech application. During the dialog session, the user interacts using an original modality and a second modality. The speech application interacts using a speech modality only. A set of conditions indicative of interaction problems using the original modality can be inferred. Responsive to the inferring step, the original modality can be changed to the second modality. A modality transition to the second modality can be transparent the speech application and can occur without interrupting the dialog session. The original modality and the second modality can be different modalities; one including a text exchange modality and another including a speech modality.