They had to work hard for the win, coming back from being a goal twice to collect three points that put them on six points.
And they plsy Malaysia on Friday, a game that will decide if they play the final or at the least the 3/4 placing.
It was New Zealand that broke the deadlock in the 12th minute through Maks Wyndham Smith.
But Japan got back seconds later through a penalty corner converted by Keigo Tamura.
Plsy was even as both teams never really dominated proceedings.
The Kiwis nudged ahead 2-1, this time Charl Ultich doing the job by scoring in the 19th minute.
New Zealand held a 2-1 lead at halftime.
However Japan looked more dangerous upon resumption and their persistence paid dividends with Keigo Tamura scoring his second of the match in the 32nd minute.
And it got better for Japan as they went 3-2 up in the next minute , yet another penalty corner, this time Riku Tamura scoring.
Sota Ukai extended the lead via a field goal in the 44th minute.
With the clocking ticking away, Japan slowed the pace to register their second successive win.