The mainboard makes an assumption about how many ports and of which type are supported though (USB, OTG, SATA, HDMI, etc.). That is still CPU specific. Most processors will support the ports provided on the mainboard (except maybe the OTG?).The stuff on the mainboard is nothing that's tied to any CPU.
We've got a normal keyboard controller, power and charging circuit, some devices (like sensors, etc.) using standard SDIO and GPI connections, MIPI for display and stuff like USB ports where the pins go directly onto the CPU board.
Unfortunately, it does not work the other way around. You will not be able to utilize additional ports supported by the CPU unless you redesign the whole thing (mainboard & casing). You would need a backplate and daughterboard for the ports for that. That would be very challenging in such a form factor.