You cannot enter Japan only with a certificate of eligibility. You need to receive the issuance of visa at a diplomatic mission abroad by presenting your certificate of eligibility.
The Certificate does not always guarantee your entering Japan. You might be denied landing Japan if the landing examination has revealed that you obviously fail to satisfy the landing permission standards as your circumstances have changed.
(source: Immigration Services Agency of Japan)