Plans to turn the historic Imperial Sugar refinery complex into a mixed-use development have been in the works for 20 years.