Of all the criticisms levelled at Keir Starmer’s No 10 during the past few months, the one that seems to have stung the most is that he presides over a “boys’ club”. A claim that “the lads” were in charge was made by the departing Labour MP Rosie Duffield, who had not seen eye to eye with her own party for a long time, referring to male political advisers whom she believed had been throwing their weight around. The accusation was easy to dismiss when Sue Gray, Starmer’s chief of staff, was the most powerful political operator in No 10 – at the prime minister’s side during his dinner with Donald Trump, and his meeting with Ursula von der Leyen in Brussels. But the following week, Starmer demoted Gray, just a year after hiring her. In her place, he brought in Morgan McSweeney, the former chief of staff and election strategist who helped bring him to power, as well as hiring a new head of strategic communications, James Lyons, a former journalist and NHS media chief. Starmer himself is not particularly blokey, aside from a love of football and the occasional pint. And neither is McSweeney, his new chief of staff, whom colleagues say is softly spoken and charming – rather than a shouty type in the vein of the The Thick of It’s Malcolm Tucker. But the narrative began again that male aides were in the ascendancy, while women held more junior positions. At the same time, some female Labour MPs were unhappy about Gray’s removal, arguing it was ill-judged to treat a senior woman from a working class background in such a ruthless way. “Of course it was because she is – as Theresa May was once called – a bloody difficult woman, and lots of them didn’t like that,” said one of her friends. But those in Downing Street who disliked Gray and wanted the prime minister to get rid of her were not confined to men. “It was a kind of mass revolt,” says one person with knowledge of how it happened. “She didn’t have many supporters in the building by the end.” There has also been pushback against the idea of a boys’ club from insiders who point out it diminishes the work of women in Starmer’s team. Under McSweeney are two senior female deputies, Jill Cuthbertson, a political fixer and organiser, and Vidhya Alakeson, formerly Starmer’s political director. The prime minister has also brought in a female principal private secretary, Nin Pandit, who will be his key consigliere alongside McSweeney and whoever is the new cabinet secretary – potentially the first woman to hold the job. In the communications team, two senior roles are held by Steph Driver, the deputy director of communications, and Sophie Nazemi, the prime minister’s political spokesperson. In one speech to Labour staff earlier this year, Hollie Ridley, now general secretary of the party, elevated on Starmer’s watch, said the briefings about the “boys” had to stop and were undermining the efforts and recognition of women in the party. Under Starmer, women hold the roles of chancellor, deputy prime minister, general secretary of Labour, party chair, principal private secretary and many others, including half of the cabinet. Even though there is little truth in the idea of a “lads’” cabal running the government, in a post-Gray No 10 the prime minister may still need to dispel that perception by showing how much he values all those senior female voices around the table.