It's hard to think of a browser less secure than Microsoft Edge. Other than Internet explorer of course.
In fact, from a security standpoint, it's always a good idea not to use the default browser that comes with your operating system. From the point of view of an outside attacker, it's a known target to aim for.
This isn't strictly true on Linux-based systems as the browser could be a third party browser already. Something like Firefox, Opera or a Tor browser.
Although it's not recommended to completely remove Microsoft Edge from Windows 10, you can limit what it can do on the system, and thereby limit the damage that can be done by an attacker.
Go to the Windows Start menu, scroll until you find 'Microsoft Edge'
Right-click Microsoft Edge, and in the popup menu, choose More > App Settings
In the new window that appears, turn off the following App permissions
Camera > Off
Microphone > Off
Account Info > Off
Pictures > Off
Background Apps > Off
A little further down, Click 'Terminate' to close any Microsoft Edge related programs and tasks The 'Uninstall' option is greyed-out as it's pretty well embedded into the system (although I will write a guide on removing Edge to follow).
Use an alternative web browser - Everyone has their preference. Mine is currently 'Firefox' as it's one of the most secure and is updated far more regularly than others. Google Chrome is a close second, although there are some pretty damning exploits available for Chrome which makes it less secure (plus I'm not a fan of the extremely minimal User Interface).
Here is a followup on disabling Microsoft Edge in Windows 10. (Because it's a security risk, it's slow.. and many other reasons). Anyway:
First, make sure you download and setup an alternative web browser. Something like Opera, Chrome, Firefox... your choice.
Once that's done, remember that Windows won't allow you to modify files in these folders. You need to either start up with a Linux LiveCD, (See https://livecdlist.com/ubuntu/). Or, you can start up with a live Windows 10 CD, something like Hiren's Boot CD. (https://www.hirensbootcd.org/files/HBCD_PE_x64.iso)
So, assuming you boot up in a live CD and can then see what is your windows drive, navigate to [windows drive]:\Windows\SystemApps\Microsoft.MicrosoftEdge_8wekyb3d8bbwe\
Find MicrosoftEdge.exe in this window and right click it.
Create a zip archive from it. You should now have MicrosoftEdge.zip (I called mine 'MicrosoftEdge-disabled.zip', so it's obvious if I want to enable Microsoft Edge ever again - can't think why I would though.)
Delete the original MicrosoftEdge.exe and empty the bin so it's gone.
Reboot back into Windows and when anything tries to launch Microsoft Edge, it'll just fail without a dialog.
Zipping Microsoft Edge.exe in Windows 10