AkumaUCF10

So to start off I have a JZS171 crown engine swapped into my 2000 GS300. I know I know, not an IS. But seeing as there's virtually nobody with 1J GS's and tons of ppl with 1J IS's, I felt like this would be a way better place to ask questions...

First off I'm like 99% I got the wiring correct I labored over the diagrams and worked out the bugs and now have it to where it starts and runs. Here's the issue I'm dealing with: after the initial start up and rev up the engine idle will drop to around 5-700 rpms. It sounds very lopey like its cammed and will eventually die. I had to tow the car home from my shop and the only way to do it was to mess with the idle control. This issue virtually goes away once the car is warm however. It runs just fine and idle is normal for a 1J ~1000 rpm +/- 100.

Stuff I've done to the motor so far: new thermostat, new MAF, new aftermarket turbo elbow, new plugs, new plug wires, intercooler, intake, manual swapped with an A/T emulator

So I have a couple of theories...
1) I'm running an intake designed for a GS300 running the bigger plate style MAF. I have modded this plate to run the later style (same as JZS171) MAF however the sensor is itself is sitting a little bit out of the stream. Could this be enough to cause this issue? It seemed to run awful if at all when the MAF was clocked wrong.
2) The coolant temp sensor may be bad. I thought this might be an issue, however this should be ruled out because the car runs fine when warm? Thoughts?
3) There is a difference in the wiring from the JZS171 motor and the JZS160 chassis when it comes to the starter wiring. The pinout on the A16 ECU plug (second from the front of car) has a black wire in position two which is referred to on both diagrams as STA or STArt. On both diagrams this wire goes into a junction box where it meets wires dedicated to the start circuit. On the JZS171 motor this wire is tied into the TRIGGERED circuit for the starter relay, in other words once the car is started it no longer sends a voltage signal to the ECU. On the GS300 chassis this wire is tied into the ENERGIZING circuit for the starter relay, in other words even after the car is started it will send a signal to the ECU. Now I realize that this one probably doesn't make a lot of sense however it does if it leans out the car during starting as to keep the car from getting flooded. The GS (same with the IS?) is a returnless fuel system which means that even on startup the fuel pressure is 100% controlled by the fuel pump ecu (which is bypassed on my car).
4) I am not as good as I thought and the wiring is not 100% correct and this is causing the issue? I have the chassis flying leads hooked up to the right place, and I even split the brown sensor ground wire (controls the MAF, coolant temp, accelerator, throttle position, power steering pressure, outside air temp) because the JZS171 only has one and the GS needs two. I do have a few wires that are not hooked up as I do not know what they go to. A red with black stripe coming off the engine, and a blue with a yellow stripe.

I know that was long winded but I just wanna give as much info as possible so that it can help narrow stuff down. Thanks ahead of time for anyone that can assist. Anyone wanting wiring diagram help with their swap feel free to message me I'll be more than glad to help, I just ask we keep the thread on topic and not all about wiring questions. Thanks guys and cheers!