A broken gameplay element (whether it be a character, stage, item, situation, or combination thereof) is so overpowering that it ruins gameplay by supplanting other gameplay elements. A classic example is Fox in Hyrule Temple; he can outrun almost every other character on the massive stage, and can chip at the enemy by firing lasers; thus, he can easily win matches by percentage difference. This type of unfair strategy is "broken" and action is taken (in this example, banning Hyrule Temple from tournament legal stages) to prevent it from impacting competitive play.
The term "broken" is often used when describing techniques, characters, attacks, etc. in many competitive games. While the term is prevalent among many games, it is often overused. Many players are quick to label strong techniques as "broken" when in reality there are consistent counters to those techniques that have not yet been discovered (although whether one can say a technique is not broken just because of undiscovered counter-techniques is fuzzy). If a broken technique is not dealt with, the game would simply degenerate into both players trying to be the first to make use of the broken element as no other element would prove effective.