At least under PHP 5.5.18 use of this function followed by constant redefinition like:
runkit_constant_remove("CONSTANT");
define("CONSTANT", "value");
may cause the constant to go into inconsistent state: from run-to-run var_dump(CONSTANT); may return NULL while $s = CONSTANT; will cause "Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 1714381161 bytes)" (note that 1G of memory configured and it is still not enough). On some runs code behaves as expected.
I believe runkit doing some low level manipulation with pointers and perhaps it is just some form of bug. Just stay wary: fiddling around with PHP on low level likely to cause unexpected results.