Goals from Marc-Antoine Fortuné, Romelu Lukaku and Shane Long saw the Baggies run out three-nil victors to become only the second side to do the double over Saints this season.
There was further misery for the hosts as Gastón Ramírez and Danny Fox were both shown straight red cards, while Albion goalscorer Fortuné was also dismissed.
And, while Saints have only dropped one place in the Premier League standings, Lallana says the defeat has left the side looking over their shoulders.
"It was a very disappointing result, obviously," said Lallana after the match.
"Losing three-nil at home doesn't happen often, so we're deeply disappointed.
"We're going to have to work hard to put this right because it proves that we're not safe yet.
"Until it's mathematically proven that you can't go down, you are always going to have to look over your shoulder.
"That's why it's so disappointing – we could maybe have put it to bed today but it wasn't to be."
And Lallana insisted that blame for the defeat should be shared equally among the team rather than aimed at the two players who were shown red cards.
He said: "They've put their hands up and said sorry for letting the lads down, but we don't see it as letting us down. It was just the heat of the moment and these things happen.
"We were all off the pace today. That showed in our performance and the result.
"It's really disappointing because West Brom beat us at the Hawthorns and we owed them one, really. It was so easy for them today.
"They came here and their tactics were good. Both of their strikers were very, very lively and they played counter-attacking football well.
"It was what you'd call 'one of those days', but we can't just say that. We didn't play as well as we have been at home.
"We didn't play high up the pitch and we didn't press well enough or pass the ball enough. All of these factors combined into a poor performance."
This is Saints' first defeat in seven games, and comes on the back of an impressive run of results.
And Lallana says the squad are determined to put things right this week ahead of next Saturday's trip to Spurs.
"We're going to have to work very hard this week because we've got a tough game on Saturday, but we want to get a result," he said.
"We had a good performance here against West Ham and we were unlucky not to win, and then a good point at Swansea but today was very disappointing in all aspects."