You need to uncheck the "Show at launch" option at the bottom left corner of the window.

Sometimes the window shows up again probably due to a bug of OS X, when it fails to properly save your User Defaults at certain cases. Please uncheck the option again, hope it will fixit for later. Very sorry for the inconvenience, we do hope the issue will be fixed with an OS update soon.