It's important to know that if you withdraw money from an ISA, you'll lose the tax benefits on that money.
But, you don't need to keep the money in your ISA with the same manager. You can transfer ISAs to or away from different managers. The rules say that you can transfer some or all of the contents in ISAs from previous tax years, but only the total amount you've got in the current year's ISA.
Transfers made from one ISA manager to another ISA manager in a tax year will not impact on your annual ISA allowance for that tax year. So, you could open a stocks and shares ISA to put your 2019/2020 allowance into, and open a cash ISA to transfer previous years ISA allowances into. Or you could open an ISA and put your new allowance into it, and transfer old allowances into it too.
Check to make sure that the ISA you open accepts transfers.