Travelling around with English people, I am often surprised at the expressions coming out of their mouths. I consider myself reasonably well versed in English, and I will recognize a lot of cultural references or can usually infer from the context what a person means. Because non-native English speakers tend to be less capable, they speak in easier words and are therefore easier for me to understand, since I've gone through a phase myself where I would search for similar words. Different continents fall back on different patterns, eg. Spanish and Chinese people not differentiating between he or she, Italians and Polish people forgetting the 'the', and so on.
Americans are a whole other story. There's a lot of Americans that, even if they did go out and visit the world, hardly have experience talking to non-native/non-capable English speakers. So they tend to include a lot of slang, complicated words and weird expressions into the conversation that only an American or someone who lived in the US would understand. I find that Americans are almost always like this towards me when they detect that I can speak and understand English. Sometimes they suddenly worry that I'm not able to follow and try to clarify what the word 'douchebag' means after a long conversation about memes, American Football, Ice Hockey and other things Americans like.
But Brits, don't get me started on them. Their English is terrible. No one in the world understands the English of the English. You'd better speak The Queen's English damn proper, and moderate your vocabulary a lot to be understood by non-native English speakers. There's a fair bit of accents here in the UK, and all of them differ from each other quite strongly. Compare cockney versus queen's versus Scottish accent versus Welsh accent. Good luck with that, non-native person just arrived in the UK.
Accents are not the main issue though, since after a while you get used to them. What I found the most difficult to adjust to here in the UK is the huge amount of words and expressions that only someone who's lived in the UK all their lives would know about. In the case of American English some commonly used colloquialisms tend to make their way into International English, but that's not the case at all when it comes to UK English.
My girlfriend regularly throws out phrases that I cannot even begin to fathom the meaning of, even knowing exactly what the current topic is or what she's looking at.For example, the other day we saw a bunch of student hikers fully packed with backpacks, sleeping mats etc. My girlfriend's response: "Oh, they're D of E!".
..
..WHAT? What is D of E and why do you expect me to understand what that means? Speak proper English like the rest of the world! She clarified: "Duke of Edinburgh". Uh, ok? They're all Duke of Edinburgh? Then she explained that young kids in the UK do some kind of group bonding exercise thing that they get points for or whatever, and this program is named after the Duke of Edinburgh who first invented it. Right.
That's just one example, but there's tons. Here's another one: what's the connection between these words: Bap, Hoagie, Sarnie, Butty, Doorstop and Club? Answer: they're all a kind of bread dish. In Dutch we call them all bread. Eskimos and snow, I guess.
I'd talk about Bric-a-brac here too but the wiki page says enough. All this inserting strange words into a conversation is quite a palava and has left my brain in a higgledy piggledy state..