即时通信 IM Android&iOS&Windows&Mac（语音转文字）

一、功能描述

注意：说明：

二、效果展示

你可以使用本功能，实现如下所示的转文字效果：

三、接口说明

你可以调用 convertVoiceToText (Java / Swift / Objective-C / C++) 接口将语音消息转换成文本。

接口参数说明如下：

示例代码如下：

JavaSwiftObjective-CC++// 获取到语音消息的 V2TIMMessage 对象V2TIMMessage msg = messageList.get(0);if (msg.elemType == V2TIM_ELEM_TYPE_SOUND) { // 从 V2TIMMessage 中获取到 soundElem V2TIMSoundElem soundElem = msg.getSoundElem(); // 调用语音转文字，默认使用中英文识别模型 soundElem.convertVoiceToText("",new V2TIMValueCallback<String>() { @Override public void onError(int code, String desc) { TUIChatUtils.callbackOnError(callBack, TAG, code, desc); String str = "convertVoiceToText failed, code: " + code + " desc: " + desc; ToastUtil.show(str,true, 1); } @Override public void onSuccess(String result) { // 识别成功，result 为识别结果 String str = "convertVoiceToText succeed, result: " + result; ToastUtil.show(str, true, 1); } });}// 获取到语音消息的 V2TIMMessage 对象let msg = messageList[0]// 检查消息类型是否为语音if msg.elemType == .ELEM_TYPE_SOUND, let soundElem = msg.soundElem { // 调用语音转文字，默认使用中英文识别模型 soundElem.convertVoiceToText(language: "") { code, desc, result in // 识别成功，result 为识别结果 print("convertVoiceToText, code: \\(code), desc: \\(desc ?? ""), result: \\(result ?? "")") }}// 获取到语音消息的 V2TIMMessage 对象V2TIMMessage *msg = messageList[0];if (msg.elemType == V2TIM_ELEM_TYPE_SOUND) { // 从 V2TIMMessage 中获取到 soundElem V2TIMSoundElem *soundElem = msg.soundElem; // 调用语音转文字，默认使用中英文识别模型 [soundElem convertVoiceToText:@"" completion:^(int code, NSString *desc, NSString *result) { // 识别成功，result 为识别结果 NSLog(@"convertVoiceToText, code: %d, desc: %@, result: %@", code, desc, result); }];}template <class T>class ValueCallback final : public V2TIMValueCallback<T> {public: using SuccessCallback = std::function<void(const T&)>; using ErrorCallback = std::function<void(int, const V2TIMString&)>; ValueCallback() = default; ~ValueCallback() override = default; void SetCallback(SuccessCallback success_callback, ErrorCallback error_callback) { success_callback_ = std::move(success_callback); error_callback_ = std::move(error_callback); } void OnSuccess(const T& value) override { if (success_callback_) { success_callback_(value); } } void OnError(int error_code, const V2TIMString& error_message) override { if (error_callback_) { error_callback_(error_code, error_message); } }private: SuccessCallback success_callback_; ErrorCallback error_callback_;};auto callback = new ValueCallback<V2TIMString>{};callback->SetCallback( [=](const V2TIMString& result) { // 语音转文字成功，result 为转换结果 delete callback; }, [=](int error_code, const V2TIMString& error_message) { // 语音转文字失败 delete callback; });// 获取到语音消息的 V2TIMMessage 对象V2TIMMessage *msg = messageList[0];// 从 V2TIMMessage 中获取到 soundElemV2TIMElem *elem = msg->elemList[0];if (elem->elemType == V2TIM_ELEM_TYPE_SOUND) { V2TIMSoundElem *sound_elem = (V2TIMSoundElem *)elem; // 调用语音转文字，默认使用中英文识别模型 sound_elem->ConvertVoiceToText("", callback);}

四、录音上传云端

你可以调用 callExperimentalAPI (Java / Swift / Objective-C / C++) 接口的 uploadFile 能力将录音上传云端并生成 url 下载地址。

接口参数说明如下：

示例代码如下：

JavaSwifttry { JSONObject uploadParam = new JSONObject(); uploadParam.put("filePath", recordPath); uploadParam.put("fileType", 3); // 0: Unknown, 1: Image, 2: Video, 3: Audio, 4: Log V2TIMManager.getInstance().callExperimentalAPI( "uploadFile", uploadParam.toString(), new V2TIMValueCallback<Object>() { @Override public void onSuccess(Object result) { if (result instanceof String) { String fileUrl = (String) result; System.out.println("Upload successful, URL: " + fileUrl); } } @Override public void onError(int code, String desc) { System.err.println("Upload failed, code: " + code + ", desc: " + desc); } } );} catch (JSONException e) { e.printStackTrace();}let uploadParam: [String: Any] = [ "filePath": recordPath, "fileType": 3 // 0: Unknown 1: Image 2: Video 3: Audio 4: log]if let uploadData = try? JSONSerialization.data(withJSONObject: uploadParam, options: .prettyPrinted), let uploadParamStr = String(data: uploadData, encoding: .utf8) { V2TIMManager.sharedInstance().callExperimentalAPI( api: "uploadFile", param: uploadParamStr as NSObject, succ: { result in if let fileUrl = result as? String, !fileUrl.isEmpty { print("Upload successful, URL: \\(fileUrl)") } }, fail: { code, desc in print("Upload failed, code: \\(code), desc: \\(String(describing: desc))") } )}

五、云端语音转文字

你可以调用 callExperimentalAPI (Java / Swift / Objective-C / C++) 接口的 convertVoiceToText 能力将云端语音转换成文字。

接口参数说明如下：

示例代码如下：

JavaSwifttry { JSONObject convertParam = new JSONObject(); convertParam.put("url", fileUrl); convertParam.put("language", ""); // Empty means auto-detect V2TIMManager.getInstance().callExperimentalAPI( "convertVoiceToText", convertParam.toString(), new V2TIMValueCallback<Object>() { @Override public void onSuccess(Object result) { if (result instanceof String) { String text = (String) result; System.out.println("Voice to text successful: " + text); } } @Override public void onError(int code, String desc) { System.err.println("Voice to text failed, code: " + code + ", desc: " + desc); } } );} catch (JSONException e) { e.printStackTrace();}let convertParam: [String: Any] = [ "url": fileUrl, "language": "" // Empty means auto-detect]if let convertData = try? JSONSerialization.data(withJSONObject: convertParam, options: .prettyPrinted), let convertParamStr = String(data: convertData, encoding: .utf8) { V2TIMManager.sharedInstance().callExperimentalAPI( api: "convertVoiceToText", param: convertParamStr as NSObject, succ: { result in if let text = result as? String { print("Voice to text successful: \\(text)") } }, fail: { code, desc in print("Voice to text failed, code: \\(code), desc: \\(String(describing: desc))") } )}

六、实时语音转文字

录音时持续上传音频、实时返回文字，适用于“边说边出字”。无需先上传完整文件。

七、接口说明

通过 callExperimentalAPI 调用以下三个实验性接口完成识别：

结果通过 onExperimentalNotify 返回，key 为 onRecvRealtimeTranscriberResponse：

参数类型：

八、调用流程

1. 注册结果监听，创建会话并保存有效 sessionID；创建期间采集的音频先缓存。

2. 按采集顺序分批上传，按 chunkIndex 覆盖更新文本、升序拼接。

3. 停止采集，尾批上传回调到达后请求停止；停止成功后仍保留监听和会话 ID。

4. 收到 isFinished = true 后取最终文本、清理会话和监听；未收到时由收尾超时兜底。

九、示例代码

以下代码直接调用 IM SDK，按顺序使用，变量和监听器由应用持有。opusPackets 由编码器提供；录音、缓存和超时需自行实现。

十、注册监听并处理识别结果

JavaSwiftObjective-Cimport com.tencent.imsdk.v2.V2TIMManager;import com.tencent.imsdk.v2.V2TIMSDKListener;import com.tencent.imsdk.v2.V2TIMValueCallback;import java.util.HashMap;import java.util.List;import java.util.Map;import java.util.TreeMap;// 以下字段放在会话对象中，调用及状态更新在同一线程执行。private String sessionID;private final TreeMap<Integer, String> sentences = new TreeMap<>();private final V2TIMSDKListener asrListener = new V2TIMSDKListener() { @Override public void onExperimentalNotify(String key, Object param) { if (!"onRecvRealtimeTranscriberResponse".equals(key) || !(param instanceof Map)) { return; } Map<?, ?> result = (Map<?, ?>) param; if (sessionID == null || !sessionID.equals(result.get("sessionID"))) { return; } Object rawIndex = result.get("chunkIndex"); int index = rawIndex instanceof Number ? ((Number) rawIndex).intValue() : 0; Object rawText = result.get("text"); String text = rawText instanceof String ? (String) rawText : ""; if (!text.isEmpty()) { sentences.put(index, text); } StringBuilder fullText = new StringBuilder(); for (String sentence : sentences.values()) { fullText.append(sentence); } System.out.println("ASR text: " + fullText); if (Boolean.TRUE.equals(result.get("isFinished"))) { System.out.println("ASR final text: " + fullText); sessionID = null; V2TIMManager.getInstance().removeIMSDKListener(this); } }};// 创建会话前注册。V2TIMManager.getInstance().addIMSDKListener(asrListener);import Foundationimport ImSDK_Plus// 以下成员放在应用持有的 NSObject 子类中，并遵循 V2TIMSDKListener。private var sessionID: String?private var sentences: [Int: String] = [:]func onExperimentalNotify(key: String, param: NSObject?) { guard key == "onRecvRealtimeTranscriberResponse", let result = param as? [String: Any], let id = sessionID, result["sessionID"] as? String == id else { return } let index = (result["chunkIndex"] as? NSNumber)?.intValue ?? 0 let text = result["text"] as? String ?? "" if !text.isEmpty { sentences[index] = text } let fullText = sentences.sorted { $0.key < $1.key }.map(\\.value).joined() print("ASR text: \\(fullText)") if (result["isFinished"] as? NSNumber)?.boolValue == true { print("ASR final text: \\(fullText)") sessionID = nil V2TIMManager.sharedInstance().removeIMSDKListener(listener: self) }}// 创建会话前注册，self 遵循 V2TIMSDKListener。V2TIMManager.sharedInstance().addIMSDKListener(listener: self)#import <ImSDK_Plus/ImSDK_Plus.h>// 声明在应用持有的 NSObject<V2TIMSDKListener> 子类中。@property(nonatomic, copy) NSString *sessionID;@property(nonatomic, strong) NSMutableDictionary<NSNumber *, NSString *> *sentences;- (void)onExperimentalNotify:(NSString *)key param:(NSObject *)param { if (![key isEqualToString:@"onRecvRealtimeTranscriberResponse"] || ![param isKindOfClass:NSDictionary.class]) { return; } NSDictionary *result = (NSDictionary *)param; if (!self.sessionID || ![self.sessionID isEqualToString:result[@"sessionID"]]) { return; } NSNumber *index = result[@"chunkIndex"] ?: @0; NSString *text = result[@"text"] ?: @""; if (text.length > 0) { self.sentences[index] = text; } NSArray<NSNumber *> *indices = [self.sentences.allKeys sortedArrayUsingSelector:@selector(compare:)]; NSMutableString *fullText = [NSMutableString string]; for (NSNumber *key in indices) { [fullText appendString:self.sentences[key]]; } NSLog(@"ASR text: %@", fullText); if ([result[@"isFinished"] boolValue]) { NSLog(@"ASR final text: %@", fullText); self.sessionID = nil; [[V2TIMManager sharedInstance] removeIMSDKListener:self]; }}// 创建会话前初始化并注册。self.sentences = [NSMutableDictionary dictionary];[[V2TIMManager sharedInstance] addIMSDKListener:self];

十一、创建实时识别会话

旧会话结束后才能重新启动，并需重新注册监听、清空句子缓存。

JavaSwiftObjective-Csentences.clear();Map<String, Object> startParams = new HashMap<>();startParams.put("language", "zh");V2TIMManager.getInstance().callExperimentalAPI( "startRealtimeTranscriber", startParams, new V2TIMValueCallback<Object>() { @Override public void onSuccess(Object result) { String id = result instanceof String ? (String) result : ""; if (id.isEmpty()) { V2TIMManager.getInstance().removeIMSDKListener(asrListener); System.err.println("ASR sessionID is empty"); return; } sessionID = id; System.out.println("ASR started: " + id); } @Override public void onError(int code, String desc) { V2TIMManager.getInstance().removeIMSDKListener(asrListener); System.err.println("ASR start failed: " + code + ", " + desc); } });sentences.removeAll()V2TIMManager.sharedInstance().callExperimentalAPI( api: "startRealtimeTranscriber", param: ["language": "zh"] as NSDictionary, succ: { [weak self] result in guard let self else { return } guard let id = result as? String, !id.isEmpty else { V2TIMManager.sharedInstance().removeIMSDKListener(listener: self) print("ASR sessionID is empty") return } self.sessionID = id print("ASR started: \\(id)") }, fail: { [weak self] code, desc in if let self { V2TIMManager.sharedInstance().removeIMSDKListener(listener: self) } print("ASR start failed: \\(code), \\(desc ?? "")") })[self.sentences removeAllObjects];[[V2TIMManager sharedInstance] callExperimentalAPI:@"startRealtimeTranscriber" param:@{@"language": @"zh"} succ:^(NSObject *result) { NSString *id = [result isKindOfClass:NSString.class] ? (NSString *)result : @""; if (id.length == 0) { [[V2TIMManager sharedInstance] removeIMSDKListener:self]; NSLog(@"ASR sessionID is empty"); return; } self.sessionID = id; NSLog(@"ASR started: %@", id); } fail:^(int code, NSString *desc) { [[V2TIMManager sharedInstance] removeIMSDKListener:self]; NSLog(@"ASR start failed: %d, %@", code, desc); }];

十二、上传 Opus 音频

取得有效 sessionID 后分批上传；尾批的成功或失败回调到达后，再请求停止。

JavaSwiftObjective-C// opusPackets: List<byte[]>，每个元素为一个完整的 Opus packet。if (sessionID != null && !opusPackets.isEmpty()) { Map<String, Object> feedParams = new HashMap<>(); feedParams.put("sessionID", sessionID); feedParams.put("dataList", opusPackets); V2TIMManager.getInstance().callExperimentalAPI( "feedRealtimeAudioData", feedParams, new V2TIMValueCallback<Object>() { @Override public void onSuccess(Object result) { } @Override public void onError(int code, String desc) { System.err.println("ASR feed failed: " + code + ", " + desc); } } );}// opusPackets: [Data]，每个元素为一个完整的 Opus packet。if let id = sessionID, !opusPackets.isEmpty { V2TIMManager.sharedInstance().callExperimentalAPI( api: "feedRealtimeAudioData", param: ["sessionID": id, "dataList": opusPackets] as NSDictionary, succ: { _ in }, fail: { code, desc in print("ASR feed failed: \\(code), \\(desc ?? "")") } )}// opusPackets: NSArray<NSData *>，每个元素为一个完整的 Opus packet。if (self.sessionID.length > 0 && opusPackets.count > 0) { [[V2TIMManager sharedInstance] callExperimentalAPI:@"feedRealtimeAudioData" param:@{@"sessionID": self.sessionID, @"dataList": opusPackets} succ:^(NSObject *result) { } fail:^(int code, NSString *desc) { NSLog(@"ASR feed failed: %d, %@", code, desc); }];}

十三、请求停止并等待尾部结果

先停采集、上传尾批再调用本接口。成功回调中不清理 sessionID 或监听器，继续等待 isFinished。

JavaSwiftObjective-Cif (sessionID != null) { Map<String, Object> stopParams = new HashMap<>(); stopParams.put("sessionID", sessionID); V2TIMManager.getInstance().callExperimentalAPI( "stopRealtimeTranscriber", stopParams, new V2TIMValueCallback<Object>() { @Override public void onSuccess(Object result) { System.out.println("ASR stop acknowledged; waiting for isFinished"); } @Override public void onError(int code, String desc) { System.err.println("ASR stop failed: " + code + ", " + desc); sessionID = null; V2TIMManager.getInstance().removeIMSDKListener(asrListener); } } );}if let id = sessionID { V2TIMManager.sharedInstance().callExperimentalAPI( api: "stopRealtimeTranscriber", param: ["sessionID": id] as NSDictionary, succ: { _ in print("ASR stop acknowledged; waiting for isFinished") }, fail: { [weak self] code, desc in print("ASR stop failed: \\(code), \\(desc ?? "")") if let self { self.sessionID = nil V2TIMManager.sharedInstance().removeIMSDKListener(listener: self) } } )}if (self.sessionID.length > 0) { [[V2TIMManager sharedInstance] callExperimentalAPI:@"stopRealtimeTranscriber" param:@{@"sessionID": self.sessionID} succ:^(NSObject *result) { NSLog(@"ASR stop acknowledged; waiting for isFinished"); } fail:^(int code, NSString *desc) { NSLog(@"ASR stop failed: %d, %@", code, desc); self.sessionID = nil; [[V2TIMManager sharedInstance] removeIMSDKListener:self]; }];}

十四、结果处理与异常收尾

·停止成功不代表结果已全部返回，以 isFinished = true 为准，并设置收尾超时（如 800 ms）兜底。空 text 不覆盖已有文本。

·只处理当前 sessionID 的结果。若取消后启动回调才返回新的 sessionID，应立即停止该会话。

·失败或退出时停止采集并移除监听。

十五、语言支持

目前支持识别的目标语言如下：