Questions regarding Opus integration in Android using JNI

Questions regarding Opus integration in Android using JNI

I am trying to integrate the Opus codec into my Android application, which
is used to stream voice from one handset to the other and vice-versa. The
communication works flawlessly without any compression, but uses a lot of
bandwidth (~500kbps per stream) and hence the need for Opus.
Since the Opus source is available in C, I've made a JNI library out of it
and am trying to use it in my code. Roughly speaking, this is the flow -
record audio from the mic using AudioRecord, pass the byte stream to Opus
to encode (and compress), send the result to the other end, decompress it
over there again using Opus and send the output to the speaker. I
currently have the following issues -
The opus_encode function requires a frame_size which is defined as "Number
of samples per channel in the input signal". How do I know what value to
pass here? The recording is done through a AudioRecord in android which
simply returns a byte stream.
The opus_decode function also requires the frame_size. I'm assuming we
would use the same value as the encode function here? Right now I'm using
a constant 2880.
Staying on the decode function, it returns the "Number of decoded
samples". When passing data to the speaker, how do I know how many bytes
of the input buffer used in the function should be passed? Basically, how
do I convert the number of decoded samples to number of bytes? Right now
I'm creating a big buffer, and just passing the entire thing to the
speaker.
In both encode and decode, I'm not sure about the datatypes being passed.
On the android end I have to send and receive byte buffers, but the encode
function takes in opus_int16* (short) and the decode function returns
unsigned char*.
The application kind of runs right now. On one phone I hear a lot of
stuttering, but I do some hear some limited audio coming out of both ends,
with a massive lag (~few seconds). Perhaps I should be running the
encoding/decoding process in its own thread.. However, I still feel that
I'm not using the Opus functions correctly.
Relevant code -
[SENDING THREAD]
try{
int minBufSize = AudioRecord.getMinBufferSize(sampleRate,
channelConfig, audioFormat);
AudioRecord recorder;
recorder = new AudioRecord(MediaRecorder.AudioSource.MIC,
sampleRate, channelConfig, audioFormat, minBufSize);
recorder.startRecording();
InetAddress IPAddress = InetAddress.getByName(ipAddress);
byte[] recordedData = new byte[minBufSize];
while (status == true)
{
// Get the voice data
recorder.read(recordedData, 0, recordedData.length);
// Try the opus encoder
byte[] compressedData = new byte[minBufSize];
int compressedLength;
compressedLength = opusWrapper.encode(recordedData,
minBufSize, compressedData, minBufSize);
byte[] trimmedCompressedData = new
byte[compressedLength];
System.arraycopy(compressedData, 0,
trimmedCompressedData, 0, compressedLength);
// Send the packet through UDP
DatagramPacket sendPacket = new
DatagramPacket(compressedData, compressedData.length,
IPAddress, port);
socket.send(sendPacket);
}
}
[RECEIVING THREAD]
try {
int minBufSize = AudioRecord.getMinBufferSize(sampleRate,
channelConfig, audioFormat);
byte[] data = new byte[0];
while(status == true) {
DatagramPacket packet = new
DatagramPacket(buffer,buffer.length);
socket.receive(packet);
// Try the opus decoder
byte[] uncompressedData = new byte[minBufSize*2];
int uncompressedLength;
uncompressedLength =
opusWrapper.decode(packet.getData(),
uncompressedData);
Log.d(TAG_RECEIVER, "Decoded samples: " +
uncompressedLength);
speaker.write(uncompressedData, 0,
uncompressedData.length);
}
[JNI WRAPPER]
private static final int OPUS_FRAME_SIZE = 2880;
public int encode(byte[] stream, int frameSize, byte[] output, int
maxDataBytes) throws Exception {
return opus_encode(encoder, stream, OPUS_FRAME_SIZE, output,
maxDataBytes);
}
public int decode(byte[] input, byte[] output) throws Exception {
return opus_decode(decoder, input, input.length, output,
OPUS_FRAME_SIZE);
}
[C CODE] (is this the correct way to pass and return buffers?)
inline jbyte* get_byte_array(JNIEnv* env, jbyteArray pcm) {
jboolean isCopy;
return (*env)->GetByteArrayElements(env, pcm, &isCopy);
}
inline void release_byte_array(JNIEnv* env, jbyteArray pcm, jbyte* data) {
(*env)->ReleaseByteArrayElements(env, pcm, data, 0);
}
JNIEXPORT jint JNICALL Java_com_anujjain_awaaz_OpusWrapper_opus_1encode
(JNIEnv * je, jobject jo, jlong encoder, jbyteArray stream, jint
frameSize, jbyteArray output, jint maxDataBytes) {
opus_int32 retVal;
jbyte *in, *out;
in = get_byte_array(je, stream);
out = get_byte_array(je, output);
retVal = opus_encode((OpusEncoder *)encoder, (opus_int16 *)in,
(int)frameSize, out, (opus_int32) maxDataBytes);
release_byte_array(je, stream, in);
release_byte_array(je, output, out);
return retVal;
}
JNIEXPORT jint JNICALL Java_com_anujjain_awaaz_OpusWrapper_opus_1decode
(JNIEnv * je, jobject jo, jlong decoder, jbyteArray input, jint
inputLength, jbyteArray output, jint frameSize) {
opus_int32 retVal;
jbyte *in, *out;
in = get_byte_array(je, input);
out = get_byte_array(je, output);
retVal = opus_decode((OpusDecoder *)decoder, in,
inputLength,(opus_int16 *)out, frameSize, 0);
release_byte_array(je, input, in);
release_byte_array(je, output, out);
return retVal;
}
I have removed a lot of extra code from the blocks above, so please excuse
any obvious mistakes or omissions.