The plot against Boris Johnson could hardly be more different from the sound and fury that characterised the one against Theresa May, which took place in crammed and sweaty meeting rooms and with public denunciations in front of amassed journalists. Instead, those hoping to oust this prime minister tend to meet in pairs over bottles of wine in the Adjournment cafe or catch a quiet word in a corridor. Nothing appears to have been coordinated on the day that many had earmarked for making their final decision on the prime minister’s future, the day of the publication of Sue Gray’s report. But MPs keeping close tally said they believed at least three more letters had been given to Sir Graham Brady, the chair of the Tory 1922 Committee, asking for a vote of no confidence. Multiple MPs who spoke to the Guardian said Johnson’s future was not secure. “I think this is the day it really hit home to some of my colleagues that we are on the brink of losing the next election,” said one former minister. “If we lose Wakefield badly I think that will prove this guy is not a winner any more. We are losing trust with the public and we will need to make some very tough decision in the coming months. We cannot make [those decisions] with any support or authority without that trust.” Another put it even more bluntly. “If we have Johnson at the next election then my voters will vote Lib Dem. If we don’t, they won’t.” An ex-minister predicted that others would send letters within days, but said there would be no organised coup. The backbench “greybeards” who rarely sought publicity were the ones reaching the end of their tether, the source said. “We are very definitely on the road to 54,” they said – a reference to the number of letters needed to trigger a leadership vote. On Wednesday afternoon, Julian Sturdy, the MP for York Outer, posted a statement on Twitter calling for Johnson to quit. Two others have not gone public. One MP who had spoken to Sturdy over the past two days said they had heard no hint of what he was planning – evidence, they suggested, that most rebels were acting alone, disorganised and disparate. Many have been burned by the experiences of trying to oust May. One MP who went public with a letter against May said they were not prepared to take that personal hit again by revealing their public position on Johnson. “A lot of us have been burned before – but also that means no one is telling anyone what they are thinking.” Another insisted they were working “with every breath in my body, with every fibre of my being” to get Johnson out of Downing Street. This MP has not publicly called for Johnson to go. Others with higher profiles, such as ex-ministers, said they were wary of being categorised only as a “Tory rebel” – a label that would stop them having a meaningful impact on other topics. Johnson’s public persona has been bullish about his survival, but inside No 10 one source admitted there were nerves among senior staff. Gray’s report arrived on the prime minister’s desk two hours late on Wednesday – a printed copy arrived just after 10am. Johnson was holed up in his study with his director of communications, Guto Harri, and chief of staff, Steve Barclay. Those who were present as the prime minister flicked through the pages say he was shaking his head, but particularly when it came to the part describing how staff had insulted a security guard during a booze-filled bash. Still, a plan was crystallising that Johnson could successfully extricate himself from the worst of the behaviour detailed in the report. One “red wall” backbencher who has wavered over Johnson’s future admitted it was “very hard to read the mood” when Johnson finished making his statement to MPs. Departing the chamber, another MP said they believed it would take several days to sink in with colleagues. The intention was to make Wednesday the final show of genuine contrition, but by the end of the day many Conservative MPs said they were vexed yet again by the prime minister’s inability to strike the right tone. Of Johnson’s key critics, several stayed away or decided they had seen enough even before 20 minutes of exchanges were up. Theresa May, the former prime minister, sat poised in her usual seat, but clearly decided it was not the moment for another intervention. Steve Baker, too, who had previously called for the prime minister to go, called it a day at about the same time that May departed. Mark Harper, who has also been public about how much the saga has appalled him, stood upright at the bar of the house, never taking his eyes off Johnson. Only Tobias Ellwood, a former defence minister, was prepared to take on the role of the Cassandra, appealing directly to Conservative MPs’ self interest by warning they were on the road to losing the next election. They heckled him. “Can we continue to govern without distraction given the erosion of the trust with the British people?” he said, turning to the benches beside him. “If we cannot work out what we’re going to do, then the broad church of the Conservative party will lose the next general election.” Johnson’s final outing of the day was to address Conservative MPs in a private meeting of the 1922 Committee. In the privacy of committee room 14, Johnson ribbed the MP Harriett Baldwin who suggested banning booze in No 10 – saying the UK would not have won the second world war without it. “He cannot help but play the after-dinner speaker and it pains me my colleagues just watch it like clapping seals,” one MP said. Few of his public critics showed their faces, though one who lingered in the door said even at the threshold he was questioning himself why he was there. “For old time’s sake?”