This right here says "You did it wrong". Which is (to me) a clear accusation of 'Bad DM', and the first thing you said to ProsecutorGodot.
That may not be your intent, but it was what you said. You didn't start by saying "Well if you didn't explain something to them they might have been confused as to what was happening and why it was happening". You said 'That isn't how it works'. Isn't how what works? YOUR game?
And as was pointed out it wasn't "a random wall" it was a thought out obstacle added to a BBEG encounter to try to keep the players from trivializing it... Which they did anyways.
Edit: Also your players may be toddlers, but not all are. I give my players puzzles that are occasionally based on things like specific sequences of prime numbers and the Fibonacci sequence. I actually did my best once to make a map with Fibonacci Spiral (points indicated by the things of interest in the Dungeon all laid on the spiral with one point that marked the secret door) that pointed them towards a secret room. They found it, but by searching for a secret door because they were suspicious of the deadend not because they saw the pattern in the map.
Also I introduce interesting mechanics all the time to keep them in their toes. It does take the right group though. Hence my comment earlier about communication and expectation being the real issue.