If you always want access to the taskbar, even when other windows are maximized to the full screen, you need to set the taskbar to always stay on top of other windows. You can still set the taskbar to automatically hide when it's not in use, but it's got to stay on top of other windows if you want access to it when other windows are maximized.
To always keep the taskbar on top of other windows, start by right-clicking on a blank area of the taskbar to bring up the context menu. Click on Properties.

