Floyd Shivambu who was the deputy president left the party to join former President Jacob Zuma's uMkhonto weSizwe (MK) party.