Conference Contacts
Clicking on Contacts will show you a list of current shared and User specific contacts.

Clicking on + Add New will let you add a new contact, choose the user type and the ownership of the contact.
To delete a contact click on Manage and then Delete. Deleting a shared contact will delete it for all users.