One of the options in WordPress you may not yet know about is your screen options. When in your Dashboard, look to the top right underneath your name, there should be two pull down menu options; Screen Options and Help. When you click on the Screen Options tab, you should see something like this:
As you try the various options, you'll see them be added or removed in real time. You'll notice different options if you do this in the main Dashboard area, vs being "Add New" posts or "Add New" page, so try the screen options in each of these. Here is how it will look under "Add New" posts for example:
You can see the options are different, and as you click on add or delete, you should see the option be removed from your screen. If you don't see any windows being removed or added, you may have to scroll down if you have a lot going on — on your screen. For example, I'm using a plugin called WordPress SEO by Yoast that is quite large. I don't see the Discussion widget get added or removed unless I scroll beneath the WordPress SEO window.