Fort Howard was a military target in 1814 when british troops landed in Fort Howard to burn and capture Baltimore. The land was used as a army post until 1940. They removed 80 buildings to make the hospital. They did keep a few old army buildings as nurses and doctors homes as well as a infirmary. Fort Howard Veterans Hospital was opened in 1943. It had 377 beds, and was mean't to do general medicine and surgical procedures. It was closed in 2002 due to structural building issues.
It seems pretty boarded up, but from google maps there are not any fences, so you might be able to find a way in.