Hannibal wanted you to go to the Lincoln Memorial to check for super mutants. If you travel there, you'll find that someone's been holding back the super mutants for awhile. It seems that some slavers are lying in wait, expecting the slaves to rally at the memorial.
If you stay off the steps of the memorial, they won't shoot at you. You can visit their leader, Leroy Walker, in the little office area. He wants you to find Hannibal for them and help crush the slave sanctuary.

click to enlarge
So we have a choice.
Help Hannibal
Helping the slaves is straightforward. Pull out a good weapon (I used Lincoln's Repeater for the poetic justice) and gun down the slavers. There are several of them guarding the Lincoln Memorial, but they should succumb pretty quickly to headshots. The only real trick is avoiding the traps at the top of the memorial. They have a lot of powerful stuff up there. If you want to be more creative, you could also try telling them that the slaves are hiding in Old Olney. This should be a speech option, if you've visited the real Old Olney before. The deathclaws will rip them apart. Don't just tell them that they are hiding near it, since this seems to signal out the Temple of the Union.
Once they're dead, just report in to Hannibal and give Caleb the poster. They'll set off on their journey.
It will take them a long time to get there. You might have to wait for a few in game days. It takes them forever to actually walk there though, if you follow them. They'll need a lot of protection and a lot of patience. I suggest you just let the game's fast travel system protect them. They'll settle in once they get there.
Help the Slavers
This is easy too. Just tell them where the Temple of the Union is. They'll move into position outside of it in a day or two. Report in and watch the slavers defeat the small guard force at the Temple. You can also do the killing yourself, if you want to be more proactive about it. Once they're all dead, the slavers will thank you and head off.