Most non-natives aren't confident when they speak English. Especially when they don't use English on a day-to-day basis.
They are always worried about grammar and vocabulary mistakes that may arise during the communication process. And therefore they look and feel uncomfortable especially when engaged in professional interaction.
But the audience may not always be aware of that and may think the speaker's hesitations are related to his or her lack of knowledge in the subject, rather than to language difficulties.
That means you may look like not knowing what you're talking about even when you're a leading expert in your field.
This is what a typical problem regarding inter-ethnic miscommunication look like.