Question: The prime minister's recent speech and your announcement on social exclusion have been rather drowned out by recent events at the top of the party – how can you express the need for this action plan most forcefully?

Hilary Armstrong: It's not inevitable that children who are born into families where there are real problems end up in trouble. And it's because we believe it's not inevitable that we know there are things that can be done.

The whole plan is characterised by early identification and early intervention.

Question: What problems do you foresee?

Hilary Armstrong: The problem with this is no government has sought to specifically tackle the bottom 2.5 per cent.

If you go back to Beveridge, he constructed a welfare state around work and what we have done very successfully is get over two million more people into work. We've used the New Deal programmes to get people who had been out of work for a long time back into the labour market, and those people who were able to get into the labour market have benefited from the minimum wage and from tax credits.

What has been done is in a sense separate them even further from this group, which are people who have been largely in generations of workless households, where frequently the children will have a poor attendance record at school, they won't turn up and get the benefit of Sure Start, they leave school without qualifications.

By the time they're 15 they will normally have a whole range of other problems. Maybe smoking, maybe substance abuse, alcohol in particular. They may have been involved in the criminal justice system by then, they may have been in care.

This is that really-difficult-to-reach group who've been left behind. And so what we're really trying to do is actually find ways of engaging with them. The early identification of where there are problems and then the early intervention itself is so that we can give them the opportunity to find a way which doesn't mean there are real crises in their lives when the state has to come in because things are falling apart.

Question: There must be people who don't want the state involved? How do you deal with them?

Hilary Armstrong: There are. However, when we're talking about the earliest intervention, midwives with pregnant women, the numbers of pregnant women who would not get involved with a midwife – that's certainly not flagged up as a problem.

And that's why we're choosing a health-led model for the earliest intervention, because they tend to be the professionals who are most trusted, who people really do believe it's going to be helpful to be engaged with them.

And all the evidence is that they do get engaged at that level, and they benefit enormously. The research on the nurse/family partnership model is striking in terms of how successful it is.

But the state does have already mechanisms to intervene even if people don't want them to, so if you really do feel a child is at risk of abuse and there's evidence there has been abuse in the past and they're refusing to take any help or support, then those children at the moment would be removed from that home.

And what we're trying to do is give different sorts of lifelines to people. If you think about when things are beginning to get really bad in the family, and the family is creating mayhem in the street, there is always the threat that if you don't behave you'll be evicted.

So the challenge then is come and work with us, come into one of the family intervention centres and we'll be able to protect a tenancy and we'll be able to make sure the children don't go into care.

But yes, there has to be a response, there's no doubt about it. Otherwise things will spiral down.

I believe it's possible and we've got to get the offer right, but we've also got to make sure there's challenge there so that we're not making the family totally dependent on the external support, that we actually give them the skills, the ability, to handle their own problems so they're going to be able to make their own way in the world.

Question: Why now?

Hilary Armstrong: Because we've done so much over the last nine years. We got involved with the 'respect' agenda, with anti-social behaviour, dealing with problems in streets.

We did that because virtually every MP would come and say that's a really big issue in their patch. For a while it was the biggest issue on their patch. And so the government has done two things over the last nine years in the sense of building up to this.

The first is we have tackled poverty and injustice, so we've lifted 800,000 children out of poverty, we've lifted two million pensioners out of poverty, through the combination of financial support, of work and improved public services.

We've made it possible for the bottom two-fifths of the population to improve their incomes at a higher rate than the rest of the population. But that has exposed this group, a group that simply hadn't got on board but remain workless without any qualifications, and then develop a whole range of other problems that are a nightmare for them and frequently a nightmare for their neighbours too.

As we have invested in public services that are universal, we've found that those universal services are sometimes simply not reaching this group.

As public service reform has been driven forward so we are able to more effectively come back to this group. They haven't been changed, their lives haven't been improved sufficiently by the mainstream services and what we've done on New Deal and so on. So we've now got to have a new drive which is a deep drive into the problems, through personalising services.

One size doesn't fit all, and we need that personalisation to make the difference.

Question: Does this involve asking voluntary organisations to do some of this work?

Hilary Armstrong: We're saying that on occasion, because people have had a bad experience of the state, for some services its right to look to the voluntary sector.

Another of the groups that we're looking at and that we talk about in the action plan are the very complex, chaotic adults who may have been in prison, may have mental health problem, may also have an addiction, may be homeless or in danger of making themselves homeless very soon. They turn up at every agency going, so they get patched up when they go to A&E and somebody sends them to the mental health support unit. They say we can't do anything because they've got a drink problem or a drug problem, and they go there and they say we can't do anything until somebody addresses their mental health problem.

So you get that sort of chaos. We are going to set up some pilots to look at whether we can attract some voluntary sector organisations into seeking to manage them as individuals more effectively so they can begin to stabilise their lives a bit more, and just begin to negotiate their way around the complexity of their problems, and I think it is mainly the voluntary sector that will fulfil that role.

And in the action plan we do identify some system reform, some of that would help us firstly identify good practice wherever that was, then help commissioners so the commissioners know what to look for and know who is out there that might provide them with the answer.

So commissioners become more confident about using a service that is in the voluntary sector for a particular type of service, or using even the private sector for another part of the service - but doing it all in a way that comes together.

So yes, we certainly do see that this sector can play a really big part, and I'm particularly committed to seeing social enterprise developed in order to do this. If you look at what the Big Issue has done in terms of working for the homeless so that they take responsibility, they sell the magazine so they get some money - but they now are looking at other ways of challenging themselves to broaden their appeal.

And because they're a social enterprise, they can have whacky ideas and try them out if they can get somebody to back them. We need that level of innovation, we need that level of commitment, but we also need those sorts of organisations working in the community.

Question: What sort of role could private companies play?

Hilary Armstrong: Well I have to tell you the private sector are already very involved in looking after children in need in this country - whether that's a good thing or a bad thing it's happening.

But if you look at the health service, bringing in private sector organisations has helped contestability and helped people see how things can be done differently. It has certainly helped get waiting lists down, and it's helped innovation in the health service, helped us bring the price of some work down, and has brought competition into the private sector in this country in health which there really wasn't for a long time.

But there will be occasions when the private sector is offering a programme. I think it's more likely to be the voluntary sector but there will be occasions when the private sector is doing it.

A lot of this is intended to feed into the spending review. You've got the local government white paper coming out later in this year, and we're working with DCLG on local area agreements and how the agreements made between local authorities and central government on what it is the local authority is looking to do. That will make sure that it includes reaching those that in the past haven't been reached.

But then through the spending review we will be looking to influence public service agreements, the agreements with departments, so that they also address this issue.

Question: Does it involve spending a lump sum of money?

Hilary Armstrong: I don't have any money, my job is to make sure the mainstream departments deliver their services in a way that actually reaches the people who often have not been reached in the past.

Then it's about how you reconfigure the service, how you individualise the service and how you just make sure that the service works.

Question: Will workers on the frontline welcome this?

Hilary Armstrong: I think they will be energised by it and will welcome it. Of course they have worries around other things like is the government going to be putting enough money into health for us all to keep our jobs, but actually some of them have said to me we know that we've got to make sure that we're working in a way that will address this, and if we haven't been in the past we know that we're going to have to reconfigure and that will make our jobs more secure.

I know I mustn't be naïve and expect everybody to say that, because any change inevitably is difficult and threatening. But overall most of them are incredibly enthusiastic and say this is what we came into the work to do.

Question: How long term is this, and how will you judge if it's successful?

Hilary Armstrong: It is long term. If you're talking about starting with children at birth, it can be 15 years before you know whether any of it has been successful.

We're basing everything on very clear, evidence-based work and the evidence elsewhere in the world is very strong. But nonetheless in order to get programmes like this really embedded into the culture throughout the country, we know that's going to take some time.

But we've got some pilot work we'll be starting immediately, and as I say what we hope to do is influence this spending review so that over the next three years following the spending review this will be an influential factor.

Question: Will this be a big issue at conference?

Hilary Armstrong: When my colleagues can hear about it, in the clamour of everything else, they're quite excited and encouraged by the agenda.

Question: As a former chief whip, what do you think of party discipline over the last week?

Hilary Armstrong: What party discipline? I think some people have taken leave of their senses, and it's very silly.