As the Commons publishes details of all MP expenses claims, new revelations have prompted the resignation of Kitty Ussher from her post in the Treasury.
Ussher, who was only appointed exchequer secretary in the reshuffle this month, resigned after it was revealed she avoided paying up to £17,000 tax on the sale of her second home.
Sarah McCarthy-Fry is moved from the education department to fill the vacant Treasury role.
The news came as the House of Commons releases edited documents outlining the expenses claims for every MP.
Key personal details have been removed, with addresses redacted in a move which makes it impossible to judge if MPs have 'flipped' their second home designations. MPs were also given several weeks to edit their own documents.
Liberal Democrat Treasury spokesman Vince Cable said the censored reports had been "compromised".
He told the BBC that much of the information about expenses would not have come out had it not been for the Daily Telegraph.
"And had it not been for my Liberal Democrat colleagues and a few other independent minded MPs from other parties, this information would have been suppressed because freedom of information powers would have been used to keep it under lock and key," Cable said.
"The compromise which has emerged is that some personal data relating to addresses has been suppressed.
"And of course, had that information never been made available to the Daily Telegraph, they would never have been able to identify the flipping of homes, the avoidance of capital gains tax, the things that were the worst abuses."
But environment secretary Hilary Benn told BBC Radio 4's Today programme that the argument for keeping bank details, phone numbers and addresses confidential was a "fair one".
He admitted that the expenses system was not capable of withstanding public scrutiny.
And he argued that Parliament was already taking steps to change the rules on MPs' allowances.
"I think the most important change that will put an end to all of this is that in future, expenses, how they are claimed, the rules and so on, will be independently regulated outside Parliament," he said.
"What has been revealed is the product of self-regulation. Members of Parliament are not the first profession and I am sure that we won't be the last, who found the limits of self-regulation. You need a different way of doing things.
"It has been very, very damaging for confidence in Parliament."
