Link to issue number:

closes #5774 closes #3644 Closes #4040

Summary of the issue:

Based on https://github.com/nvaccess/nvda/issues/577#issue-116022785

In the current situation, several settings subjects have their own dialog in NVDA. This makes configuring NVDA more complicated than necessary (i.e. you can't update all settings in one go and need to open multiple dialogs, either using the menu or using shortcut keys).

Description of how this pull request fixes the issue:

This pr combines the several configuration manager based dialogs into one single dialog. The resulting dialog has a list view with categories on the left side and a panel containing the settings on the right side.

Known issues with pull request:

From https://github.com/nvaccess/nvda/issues/577#issuecomment-309216018

• I didn't do anything about the look yet, so it might look very ugly. We probably need sighted assistance for this, so any thoughts from @feerrenrut are highly appreciated.
• Currently, all the configuration manager options (e.g. general settings, voice settings) activate the same dialog. This will probably have to go away in the future
• I would like to have a kind of heading at the top of every panel with the panel's title, but do not know yet how to accomplish this in a look friendly way. @@feerrenrut or @josephsl, ideas?
• The categories list is a list box, which can be changed into a tree view if desired. We should consider this if speech dictionaries get integrated into this dialog as well.
• The current implementation has no facilities to integrate add-ons into the system with backwards compatibility (i.e. integrating addons into the new system without any change into the add-ons code is currently not possible). Honestly, I think we should keep it that way and offer people information on how to upgrade their add-on.
• Sub dialogs and message boxes will probably break

• Changing the synthesizer or braille display without saving will make the synth or display selection go back to the initial selection when opening the dialog

Todo

• [x] Decide whether to stick to a ListBox, or should we choose a tree view anyway?
• [x] Arrange the visual lay out
• [x] Decide what to include and what not to include in the dialog (e.g. Punctuation/symbol pronunciation, speech dictionaries)

Reported by mdcurran on 2010-08-03 03:07 I have created a branch containing a new desktop keyboard layout, that also contains alternative key bindings for laptop users. I.e. a second binding for any script that uses the numpad. The branch is at: http://bzr.nvaccess.org/nvda/keyboardLayoutRefactor The advantage of unifying the laptop and desktop layouts in to one is that hopefully we will be able to maintain it much easier, and not leave the laptop layout as a "second class citizen", as some have said. There have been a few changes to the original desktop key bindings, such as moving all the settings dialog scripts to control+nvda+function keys, to free up more keys for proper numpad emulation on the right letter keys. Please try this new layout on laptops and provide comments on this ticket as to your thoughts:

• Are there any key combinations that do not work on your keyboard?
• Have I missed any key bindings?

If this new unified layout works well for people, I will then update the user guide with proper tables of keys and there alternatives. We also plan to mark up the user guide such that a key commands document can be generated straight from the user guide.

As a primary developer of NVDA, I am more than happy to make this new layout work well for people, and make sure its maintained as well as the original desktop layout. However, as noted above, there are some changes to the desktop layout etc, but I feel that this new way will be the most easiest for us to not leave the laptop layout behind. Having NVDA run on laptops/netbooks is extremely important, especially in education.

I have managed to not use the alt key at all in this layout, nore have I used any 4-key combinations (e.g. control+shift+nvda+letter). As these were two problems noted in the past.

Steps to reproduce:

1. Open Visual studio code.
2. Open a file or directory containing any text files in it such as a javascript/python project.
3. Go to the editor.
4. try navigating the text/code via arrow keys.
5. Now try typing some code in the editor quickly.
6. It will be observed that in both 4th and 5th point, NVDA laggs a lot while navigating and typing the code in the editor.
7. Now repeat the same thing with Notepad plus plus or another text editor.
8. It will be observed that NVDA doesn't lag there at all and responds very snappily.

Actual behavior:

When we navigate/read and type the text/code in visual studio code editor, NVDA laggs a lot and responds very slowly. For example, If we navigate/read the code in notepad NVDA responds/reads every line quickly but in VS Code, it doesn't happen like this. Also, If we quickly type the code, most of the times, NVDA announces the typed characters after a delay which is very annoying and problematic. It doesn't happen with other applications.

Expected behavior:

NVDA should not lag in VS Code editor and should be responsive like it is with other editors.

System configuration

NVDA installed/portable/running from source:

Installed

NVDA version:

NVDA version alpha-20793,9a4074bc

Windows version:

Tried with multiple Windows builds: Windows 10 2004, 19041.450, and latest insider too.

Name and version of other software in use when reproducing the issue:

Visual Studio Code both insider and stable.

Other information about your system:

I7 processor, 8 GB ram, Samsung SSD and also tried with multiple other systems. Same issue persists.

Other questions

Does the issue still occur after restarting your computer?

Yes, always and even after reformatting the system too.

Have you tried any other versions of NVDA? If so, please report their behaviors.

Yes, NVDA stable versions, and the same issue is being observed since months.

If addons are disabled, is your problem still occuring?

Yes, always.

Did you try to run the COM registry fixing tool in NVDA menu / tools?

Yes, but no luck

Link to issue number:

Fixes #4877. Fixes #1229.

Summary of the issue:

We want to be able to easily and accurately perform various actions (beep, play sounds, switch profiles, etc.) during speech. We also want to be able to have prioritized speech which interrupts lower priority speech and then have the lower priority speech resume. This is required for a myriad of use cases, including switching to specific synths for specific languages (#279), changing speeds for different languages (#4738), audio indication of spelling errors when reading text (#4233), indication of links using beeps (#905), reading of alerts without losing other speech forever (#3807, #6688) and changing speech rate for math (#7274). Our old speech code simply sends utterances to the synthesizer; there is no ability to do these things. Say all and speak spelling continually poll the last index, but this is ugly and not feasible for other features.

Description of how this pull request fixes the issue:

Enhance nvwave to simplify accurate indexing for speech synthesizers.

1. Add an onDone argument to WavePlayer.feed which accepts a function to be called when the provided chunk of audio has finished playing. Speech synths can simply feed audio up to an index and use the onDone callback to be accurately notified when the index is reached.
2. Add a buffered argument to the WavePlayer constructor. If True, small chunks of audio will be buffered to prevent audio glitches. This avoids the need for tricky buffering across calls in the synth driver if the synth provides fixed size chunks and an index lands near the end of a previous chunk. It is also useful for synths which always provide very small chunks.

Enhancements to config profile triggers needed for profile switching within speech sequences.

1. Allow triggers to specify that handlers watching for config profile switches should not be notified. In the case of profile switches during speech sequences, we only want to apply speech settings, not switch braille displays.
2. Add some debug logging for when profiles are activated and deactivated.

Add support for callbacks, beeps, sounds, profile switches and utterance splits during speech sequences, as well as prioritized queuing.

Changes for synth drivers:

• SynthDrivers must now accurately notify when the synth reaches an index or finishes speaking using the new synthIndexReached and synthDoneSpeaking extension points in the synthDriverHandler module. The lastIndex property is deprecated. See below regarding backwards compatibility for SynthDrivers which do not support these notifications.
• SynthDrivers must now support PitchCommand if they which to support capital pitch change.
• SynthDrivers now have supportedCommands and supportedNotifications attributes which specify what they support.
• Because there are some speech commands which trigger behaviour unrelated to synthesizers (e.g. beeps, callbacks and profile switches), commands which are passed to synthesizers are now subclasses of speech.SynthCommand.

Central speech manager:

• The core of this new functionality is the speech._SpeechManager class. It is intended for internal use only. It is called by higher level functions such as speech.speak.
• It manages queuing of speech utterances, calling callbacks at desired points in the speech, profile switching, prioritization, etc. It relies heavily on index reached and done speaking notifications from synths. These notifications alone trigger the next task in the flow.
• It maintains separate queues (speech._ManagerPriorityQueue) for each priority. As well as holding the pending speech sequences for that priority, each queue holds other information necessary to restore state (profiles, etc.) when that queue is preempted by a higher priority queue.
• See the docstring for the speech._SpeechManager class for a high level summary of the flow of control.

New/enhanced speech commands:

• EndUtteranceCommand ends the current utterance at this point in the speech. This allows you to have two utterances in a single speech sequence.
• CallbackCommand calls a function when speech reaches the command.
• BeepCommand produces a beep when speech reaches the command. This is the basis for features such as #905.
• WaveFileCommand plays a wave file when speech reaches the command. This is the basis for features such as #4233 and #4089.
• The above three commands are all subclasses of BaseCallbackCommand. You can subclass this to implement other commands which run a pre-defined function.
• ConfigProfileTriggerCommand applies (or stops applying) a configuration profile trigger to subsequent speech. This is the basis for switching profiles (and thus synthesizers, speech rates, etc.) for specific languages, math, etc.; #279, #4738, #4433, #7274, etc.
• PitchCommand, RateCommand and VolumeCommand can now take either a multiplier or an offset. In addition, they can convert between the two on demand, which makes it easier to handle these commands in synth drivers based on the synth's requirements. They also have an isDefault attribute which specifies whether this is returning to the default value (as configured by the user).

Speech priorities:

• speech.speak now accepts a priority argument specifying one of three priorities: SPRI_NORMAL (normal priority), SPRI_NEXT (speak after next utterance of lower priority)or SPRI_NOW (speech is very important and should be spoken right now, interrupting lower priority speech).
• Interrupted lower priority speech resumes after any higher priority speech is complete.
• This is the basis for fixing issues such as #3807 and #6688.

Refactored functionality to use the new framework:

• Rather than using a polling generator, spelling is now sent as a single speech sequence, including EndUtteranceCommands, BeepCommands and PitchCommands as appropriate. This can be created and incorporated elsewhere using the speech.getSpeechForSpelling function. This fixes #1229 (since it's a single sequence) and is also the basis for fixing issues such as #3286, #4874 and #4661.
• Say all has been completely rewritten to use CallbackCommands instead of a polling generator. The code should also be a lot more readable now, as it is now classes with methods for the various stages in the process.

Backwards compatibility for old synths:

• For synths that don't support index and done speaking notifications, we don't use the speech manager at all. This means none of the new functionality (callbacks, profile switching, etc.) will work.
• This means we must fall back to the old code for speak spelling, say all, etc. This code is in the speechCompat module.
• This compatibility fallback is considered deprecated and will be removed eventually. Synth drivers should be updated ASAP.

Deprecated/removed:

• speech.getLastIndex is deprecated and will simply return None.
• IndexCommand should no longer be used in speech sequences passed to speech.speak. Use a subclass of speech.BaseCallbackCommand instead.
• In the speech module, speakMessage, speakText, speakTextInfo, speakObjectProperties and speakObject no longer take an index argument. No add-ons in the official repository use this, so I figured it was safe to just remove it rather than having it do nothing.
• speech.SpeakWithoutPausesBreakCommand has been removed. Use speech.EndUtteranceCommand instead. No add-ons in the official repository use this.
• speech.speakWithoutPauses.lastSentIndex has been removed. Instead, speakWithoutPauses returns True if something was actually spoken, False if only buffering occurred.

Update comtypes to version 1.1.3.

• This is necessary to handle events from SAPI 5, as one of the parameters is a decimal which is not supported by our existing (very outdated) version of comtypes .
• comtypes has now been added as a separate git submodule.

Updated synth drivers

The espeak, oneCore and sapi5 synth drivers have all been updated to support the new speech framework.

Testing performed:

Unfortunately, I'm out of time to write unit tests for this, though much of this should be suitable for unit testing. I've been testing with the Python console test cases below. Note that the wx.CallLater is necessary so that speech doesn't get silenced straight away; that's just an artefact of testing with the console.

For the profile tests, you'll need to set up two profiles, one triggered for say all and the other triggered for the notepad app.

Python Console test cases:

# Text, beep, beep, sound, text.
wx.CallLater(500, speech.speak, [u"This is some speech and then comes a", speech.BeepCommand(440, 10), u"beep. If you liked that, let's ", speech.BeepCommand(880, 10), u"beep again. I'll speak the rest of this in a ", speech.PitchCommand(offset=50), u"higher pitch. And for the finale, let's ", speech.WaveFileCommand(r"waves\browseMode.wav"), u"play a sound."])
# Text, end utterance, text.
wx.CallLater(500, speech.speak, [u"This is the first utterance", speech.EndUtteranceCommand(), u"And this is the second"])
# Change pitch, text, end utterance, text. Expected: All should be higher pitch.
wx.CallLater(500, speech.speak, [speech.PitchCommand(offset=50), u"This is the first utterance in a higher pitch", speech.EndUtteranceCommand(), u"And this is the second"])
# Text, pitch, text, enter profile1, enter profile2, text, exit profile1, text. Expected: All text after 1 2 3 4 should be higher pitch. 5 6 7 8 should have profile 1 and 2. 9 10 11 12 should be just profile 2.
import sayAllHandler, appModuleHandler; t1 = sayAllHandler.SayAllProfileTrigger(); t2 = appModuleHandler.AppProfileTrigger("notepad"); wx.CallLater(500, speech.speak, [u"Testing testing ", speech.PitchCommand(offset=100), "1 2 3 4", speech.ConfigProfileTriggerCommand(t1, True), speech.ConfigProfileTriggerCommand(t2, True), u"5 6 7 8", speech.ConfigProfileTriggerCommand(t1, False), u"9 10 11 12"])
# Enter profile, text, exit profile. Expected: 5 6 7 8 in different profile, 9 10 11 12 with base config.
import sayAllHandler; trigger = sayAllHandler.SayAllProfileTrigger(); wx.CallLater(500, speech.speak, [speech.ConfigProfileTriggerCommand(trigger, True), u"5 6 7 8", speech.ConfigProfileTriggerCommand(trigger, False), u"9 10 11 12"])
# Two utterances at SPRI_NORMAL in same sequence. Two separate sequences at SPRI_NEXT. Expected result: numbers in order from 1 to 20.
wx.CallLater(500, speech.speak, [u"1 2 3 ", u"4 5", speech.EndUtteranceCommand(), u"16 17 18 19 20"]); wx.CallLater(510, speech.speak, [u"6 7 8 9 10"], priority=speech.SPRI_NEXT); wx.CallLater(520, speech.speak, [u"11 12 13 14 15"], priority=speech.SPRI_NEXT)
# Utterance at SPRI_NORMAL including a beep. Utterance at SPRI_NOW. Expected: Text before the beep, beep, Text after..., This is an interruption., Text after the beep, text...
wx.CallLater(500, speech.speak, [u"Text before the beep ", speech.BeepCommand(440, 10), u"text after the beep, text, text, text, text"]); wx.CallLater(1500, speech.speak, [u"This is an interruption"], priority=speech.SPRI_NOW)
# Utterance with two sequences at SPRI_NOW. Utterance at SPRI_NOW. Expected result: First utterance, second utterance
wx.CallLater(500, speech.speak, [u"First ", u"utterance"], priority=speech.SPRI_NOW); wx.CallLater(510, speech.speak, [u"Second ", u"utterance"], priority=speech.SPRI_NOW)
# Utterance with two sequences at SPRI_NOW. Utterance at SPRI_NEXT. Expected result: First utterance, second utterance
wx.CallLater(500, speech.speak, [u"First ", u"utterance"], priority=speech.SPRI_NOW); wx.CallLater(501, speech.speak, [u"Second ", u"utterance"], priority=speech.SPRI_NEXT)
# Utterance at SPRI_NORMAL. Utterance at SPRI_NOW with profile switch. Expected: Normal speaks but gets interrupted, interruption with different profile, normal speaks again
import sayAllHandler; trigger = sayAllHandler.SayAllProfileTrigger(); wx.CallLater(500, speech.speak, [u"This is a normal utterance, text, text"]); wx.CallLater(1000, speech.speak, [speech.ConfigProfileTriggerCommand(trigger, True), u"This is an interruption with a different profile"], priority=speech.SPRI_NOW)
# Utterance at SPRI_NORMAL with profile switch. Utterance at SPRI_NOW. Expected: Normal speaks with different profile but gets interrupted, interruption speaks with base config, normal speaks again with different profile
import sayAllHandler; trigger = sayAllHandler.SayAllProfileTrigger(); wx.CallLater(500, speech.speak, [speech.ConfigProfileTriggerCommand(trigger, True), u"This is a normal utterance with a different profile"]); wx.CallLater(1000, speech.speak, [u"This is an interruption"], priority=speech.SPRI_NOW)
# Utterance at SPRI_NORMAL with profile 1. Utterance at SPRI_NOW with profile 2. Expected: Normal speaks with profile 1 but gets interrupted, interruption speaks with profile 2, normal speaks again with profile 1
import sayAllHandler, appModuleHandler; t1 = sayAllHandler.SayAllProfileTrigger(); t2 = appModuleHandler.AppProfileTrigger("notepad"); wx.CallLater(500, speech.speak, [speech.ConfigProfileTriggerCommand(t1, True), u"This is a normal utterance with profile 1"]); wx.CallLater(1000, speech.speak, [speech.ConfigProfileTriggerCommand(t2, True), u"This is an interruption with profile 2"], priority=speech.SPRI_NOW)
# Utterance at SPRI_NORMAL including a pitch change and beep. Utterance at SPRI_NOW. Expected: Text speaks with higher pitch, beep, text gets interrupted, interruption speaks with normal pitch, text after the beep speaks again with higher pitch
wx.CallLater(500, speech.speak, [speech.PitchCommand(offset=100), u"Text before the beep ", speech.BeepCommand(440, 10), u"text after the beep, text, text, text, text"]); wx.CallLater(1500, speech.speak, [u"This is an interruption"], priority=speech.SPRI_NOW)

Known issues with pull request:

No issues with the code that I know of. There are two issues for the project, though:

1. All third party synth drivers need to be updated in order to support the new functionality. Old drivers will still work for now thanks to the compat code, but they get none of the new functionality. Getting third parties to do this will take some time.
2. While this PR forms the basis for a lot of functionality, it doesn't provide many user visible changes. That means merging it is risky without immediate benefit. That said, putting anything more in this PR would make it even more insane than it already is.

Change log entry:

Bug Fixes:

- When spelling text, reported tool tips are no longer interjected in the middle of the spelling. Instead, they are reported after spelling finishes. (#1229)

Changes for Developers:

- nvwave has been enhanced to simplify accurate indexing for speech synthesizers: (#4877)
 - `WavePlayer.feed` now takes an `onDone` argument specifying a function to be called when the provided chunk of audio has finished playing. Speech synths can simply feed audio up to an index and use the onDone callback to be accurately notified when the index is reached.
 - The `WavePlayer` constructor now takes a `buffered` argument. If True, small chunks of audio will be buffered to prevent audio glitches. This avoids the need for tricky buffering across calls in the synth driver if the synth provides fixed size chunks and an index lands near the end of a previous chunk. It is also useful for synths which always provide very small chunks.
- Several major changes related to synth drivers: (#4877)
 - SynthDrivers must now accurately notify when the synth reaches an index or finishes speaking using the new `synthIndexReached` and `synthDoneSpeaking` extension points in the `synthDriverHandler` module.
  - The `lastIndex` property is deprecated.
  - For drivers that don't support these, old speech code will be used. However, this means new functionality will be unavailable, including callbacks, beeps, playing audio, profile switching and prioritized speech. This old code will eventually be removed.
 - SynthDrivers must now support `PitchCommand` if they which to support capital pitch change.
 - SynthDrivers now have `supportedCommands` and `supportedNotifications` attributes which specify what they support.
 - Because there are some speech commands which trigger behaviour unrelated to synthesizers (e.g. beeps, callbacks and profile switches), commands which are passed to synthesizers are now subclasses of `speech.SynthCommand`.
- New/enhanced speech commands: (#4877)
 - `EndUtteranceCommand` ends the current utterance at this point in the speech. This allows you to have two utterances in a single speech sequence.
 - `CallbackCommand` calls a function when speech reaches the command.
 - `BeepCommand` produces a beep when speech reaches the command.
 - `WaveFileCommand` plays a wave file when speech reaches the command.
 - The above three commands are all subclasses of `BaseCallbackCommand`. You can subclass this to implement other commands which run a pre-defined function.
 - `ConfigProfileTriggerCommand` applies (or stops applying) a configuration profile trigger to subsequent speech. This is the basis for switching profiles (and thus synthesizers, speech rates, etc.) for specific languages, math, etc.
 - `PitchCommand`, `RateCommand` and `VolumeCommand` can now take either a multiplier or an offset. In addition, they can convert between the two on demand, which makes it easier to handle these commands in synth drivers based on the synth's requirements. They also have an `isDefault` attribute which specifies whether this is returning to the default value (as configured by the user).
- `speech.speak` now accepts a `priority` argument specifying one of three priorities: `SPRI_NORMAL` (normal priority), `SPRI_NEXT` (speak after next utterance of lower priority)or `SPRI_NOW` (speech is very important and should be spoken right now, interrupting lower priority speech). Interrupted lower priority speech resumes after any higher priority speech is complete. (#4877)
- Deprecated/removed speech functionality: (#4877)
 - `speech.getLastIndex` is deprecated and will simply return None.
 - `IndexCommand` should no longer be used in speech sequences passed to `speech.speak`. Use a subclass of `speech.BaseCallbackCommand` instead.
 - In the `speech` module, `speakMessage`, `speakText`, `speakTextInfo`, `speakObjectProperties` and `speakObject` no longer take an `index` argument.
 - `speech.SpeakWithoutPausesBreakCommand` has been removed. Use `speech.EndUtteranceCommand` instead.
 - `speech.speakWithoutPauses.lastSentIndex` has been removed. Instead, `speakWithoutPauses` returns True if something was actually spoken, False if only buffering occurred.
- Updated comtypes to version 1.1.3. (#4877)

Closes #7077 Closes #7429

Hi,

From 2012 to 2017, Robin Dunn and others have been working on a complete revamp of wxPython. One of the benefits of the resulting 4.0.0 release is ability to use wx modules via Python versions 2.7 and 3.5+. This will require a longer incubation period, as add-ons will need to be modified, an extensive hunt for issues must be carried out and so on.

at the moment the only major issue is wx.Yield, seen when one repeatedly taps arrow keys or certain keys quickly. For example, from Python Console, press backspace, and you may hear the error tone with a traceback that culminates in wxPython saying wx.Yield was called recursively. As for add-ons, wx.PyDeadObjectError must be changed to RuntimeError, certain wx.Center function arguments/constants have changed and so on.

Also, as part of wxPython, we also get the Six module for free, which eases transition to Python 3.x.

Thanks.

Reported by mike.reiser on 2010-08-04 17:13 Nvda should be able to handle the input of braille from a braille keyboard on displays sutch as the refreshabraille 18, and translate it back to text so that people can type in braille if they wish and it still comes out in text. This is done by system access and other readers.
Blocked by #601 Blocking #1003, #2439

Reported by jjimenez0 on 2014-04-16 05:47

EcoBraille display are very common in Spain but it doesn’t work with 64 bits systems, so they were falling obsolete. An addon for Eco Braille display would be very important in order to get working those displays in NVDA even in 64 bits operating systems.

Reported by drein on 2012-05-05 07:23 As you know, there are external tools that can be used to have a good experience with skype. the first is skypetalking, the second is gwconnect. But, all these programs have a great limitation: they don't read and announce facebook chat. If NVDA read incoming message automatically, especially when you are in the chat window, all these limits will disappear. Is it possible? Blocking #2018

Reported by fatma.mehanna on 2011-06-22 15:05 there is a problem with the arabic language. don't know if it exists before lanuching the beta or not. the problem is when we are in a skype conversation window and pressing nvda+tab to be sure of what we type before sending it, nvda reads the message correctly then repeats it but in a transposing way. i.e it reads the message again but from the end of the line. this problem also appears when we use the flat review in ms word. can't be sure if it appears in other places or not. note, we never saw this problem before. i created a log file that may help you discover the root of the problem.

First of all I'm so glad that these errors aren't caused by my HUC Braille Tables alone, because other large braille tables like "zh-tw.ctb" lead to the same results I described in issue #9973 yesterday. CC: @leonardder

Steps to reproduce:

1. Install/Create a portable copy of NVDA 2019.2 Beta-18176.
2. Now start NVDA Portable and change the braille output table to Chinese Taiwan/Mandarin ("zh-tw.ctb") and disable the checkbox "Expand to computer braille for the word at the cursor".
3. Restart NVDA with debug logging level via NVDA+Q.

Actual behavior:

This time I was able to reproduce both errors I mentioned in issue #9973 on a SSD, but I couldn't figure out if the Chinese braille table "zh-tw.ctb" (of course without the HUC Braille Tables) was completely loaded or not. Maybe you have to restart NVDA multiple times to get these errors.

Here is the first logfile with no error sound on startup. And here is the second one with an error sound and no braille output directly after startup. Always start at line 126. And please take a look at the timestamps in the second logfile, because there aren't any log entries from the beginning of the error sound until the Protocol Viewer was opened. That's so strange, therefore I repeated restarting NVDA in Debug Mode multiple times, because I thought I did something wrong. Normally NVDA should also have logged pressing ArrowRight and ArrowLeft in the Windows Explorer (Detail View) as well as the steps to open the Protocol Viewer via the NVDA menu. But that was never the case.

Expected behavior:

No errors on loading large braille tables.

• zh-tw.ctb: 1,25 MB (1,306,176 Bytes); 48,769 lines

System configuration

NVDA installed/portable/running from source:

Portable

NVDA version:

2019.2 Beta-18176

Windows version:

Win7x64

Other information about your system:

Optelec ALVA BC680 is connected via USB2. The USB1 port wasn't tested.

Other questions

Does the issue still occur after restarting your PC?

Not tested.

Have you tried any other versions of NVDA? If so, please report their behaviors.

Tests with 2019.1.1 (Portable) are pending. But I guess it's the same behavior like in issue #9973 – it works as expected. And smaller braille tables such as the Finish 8-dot Computer Braille ("fi-fi-8dot.ctb") seems to work with NVDA 2019.2 Beta-18176 as expected too. But deeper tests are pending here too.

Reported by mreiser on 2013-02-09 17:10 Since Outlook2007, it has become nearly impossible to read HTML messages in outlook with nvda and other screen readers. With jaws 14, they are somehow able to virtualize the message so that these messages so that they can be read once again. NVDA should provide a way to do this if possible.
Blocking #1968, #3871, #4187, #4188

Is your feature request related to a problem? Please describe.

NVDA Remote currently relies on a lot of monkeypatching to intercept braille, speech, gestures, tones and waves from the controlled system. #14503 intends to fix most of this, except for speech.

Speech is a difficult thing. Historically. NVDA Remote patched methods on the synth directly, e.g. speak, cancel. With speech refactor becoming available, this was changed to patch methods on the speech manager. This had several advantages, including:

1. Indexing and callback commands could be handled by the controller. This means that beeping in the middle of a sentence would work correctly, as the beep callback is executed by the controller
2. It was no longer necessary to patch the synthesizer

However, there are some drawbacks and current shortcomings:

1. NVDA Remote must mess around with the speech state. speech.speech._speechState.beenCanceled is set to False before NVDA remote can speak
2. Cancellable speech does not work

Describe the solution you'd like

I think that for NVDA Remote, the current approach should be preferred over the older approach where the controller relies on controlled system indexing. In other words, shortcoming 2 does not outweigh advantage 1. To solve shortcoming 1, it might be necessary to use speech.speak, speech.cancelSPeech and speech.pauseSPeech directly. On the other hand, for #3564, it makes more sense to work with a virtual synth driver, since the controlled system (or server) doesn't require local speech output. In that case, it's fine if the controlled system does the indexing and cancellable speech should work as well.

That said, we should solve shortcoming 1, e.g. NVDA Remote should be able to queue speech without fiddling with the internal speech state. Now I'm not very comfortable with this new approach yet. Therefore I hope @feerrenrut can share his ideas about this.

To avoid monkey patching, we need several extension points to cover both approaches:

• synthDriverHandler.synthChanged: To perform actions when a new synth is selected
• An extension point for speak. This can also be used for the speechViewer
• An extension point vor cancel
• An extension point for pause.

Additional requirements:

• Consider moving the filtering of redundant LangChangeCommand objects and handling of character commands, character and speechdict processing from speech.speech.speak to speech.manager.SPeechManager._processSpeechSequence. Note that it would make sense if a controller can use own preferences for language switching, symbol level, etc.

I'd love thoughts from @feerrenrut, @ctoth and @tspivey as well as other interested people.

Steps to reproduce:

1. Ensure to use a NVDA alpha (last, previous... I estimate from November at least);
2. open preferences, Document formatting, and disable landmarks reporting;
3. go i.e. to issues page;
4. press 3 to find h3 Footer navigation.

Actual behavior:

NVDA says (voice only, partially translated) "Footer navigation landmark Footer navigation level 3".

Expected behavior:

The usual, less verbose "Footer navigation level 3".

System configuration

NVDA installed/portable/running from source:

Installed.

NVDA version:

NVDA alpha-27446,6dd90020.

Windows version:

Windows 10 64-bit 22H2.

Name and version of other software in use when reproducing the issue:

Firefox 108.0.2, Google Chrome 108.0.5359.125.

Other questions

Does the issue still occur after restarting your computer?

Yes.

Have you tried any other versions of NVDA? If so, please report their behaviors.

Tested with NVDA 2022.4 portable, no issue.

If NVDA add-ons are disabled, is your problem still occurring?

Yes.

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

Not tested.

Steps to reproduce:

Use NVDA over time, seems to have started occurring after Windows 11 22h2 and NVDA 2022.4 betas.

Actual behavior:

NVDA menu, accessed by nvda+N does not read the submenu. See attached log for details on errors that seem to be generated. The NVDA menu is accessible using object nav, but invoking using nvda+enter does not work. To exit, you have to mouse click exit or restart NVDA using the NVDA hotkey.

Expected behavior:

NVDA menu appears and does not become unavailable when using the NVDA+N key. I cannot get speech viewer information for this behavior.

NVDA logs, crash dumps and other attachments:

nvda-error.log

System configuration

NVDA installed/portable/running from source:

NVDA installed

NVDA version:

2022.4

Windows version:

Windows 11 22H2

Name and version of other software in use when reproducing the issue:

varies

Other information about your system:

Other questions

Does the issue still occur after restarting your computer?

No, not right away, either restarting NVDA or restarting the computer brings back the menu. Te menu is diabled after a time however.

Have you tried any other versions of NVDA? If so, please report their behaviors.

Older versions did not exhibit this behavior

If NVDA add-ons are disabled, is your problem still occurring?

I disabled addons and received the same result.

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

yes

Steps to reproduce:

Given the following html:

<body>
  <h1>Heading 1</h1>
  <h2>Heading 2</h2>
  <figure aria-label="Label for figure" aria-description="A Description">
    <p aria-hidden="true">I'm paragraph text</p>
  </figure>
</body>

NVDA will not read the label or description for the figure element. This appears to be due to the child <p> having aria-hidden="true". When the aria-hidden attribute is removed NVDA will read the figure's label and description as expected.

Also, changing the <figure> to a <div> with a role="figure" reads as expected, so the following will read the label and description while properly ignoring the child <p>:

<body>
  <h1>Heading 1</h1>
  <h2>Heading 2</h2>
  <div aria-role="figure" aria-label="Label for figure" aria-description="A Description">
    <p aria-hidden="true">I'm paragraph text</p>
  </div>
</body>

Actual behavior:

The <figure> label and description attributes are not read when there is a child element that has aria-hidden="true"

Expected behavior:

NVDA should read the label and description attributes and ignore the hidden child elements using the <figure> element.

NVDA logs, crash dumps and other attachments:

System configuration

NVDA installed/portable/running from source:

NVDA version:

2022.4.0.27401

Windows version:

Version 10.0.19044 Build 19044

Name and version of other software in use when reproducing the issue:

Tested in Firefox and Chrome

Other information about your system:

Other questions

Does the issue still occur after restarting your computer?

Yes

Have you tried any other versions of NVDA? If so, please report their behaviors.

No

If NVDA add-ons are disabled, is your problem still occurring?

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

Start the dev cycle for the 2023.2 release. This won't be a compatibility breaking release.

Complete:

• [x] New section in the change log.
• [x] Update NVDA version in master

On merge:

• [ ] Update auto milestone ID

Hi,

This stems from work done in Windows App Essentials add-on:

Background:

NVDA can recognize modern IME's introduced in Windows 10, specificlaly with Version 2004 (May 2020 Update). Among other things, it allows NVDA to handle IME's as in IAccessible implementation from the past. Windows 11 does come with modern keyboard based IME interface but both gain focus and element selected events are fired when IME candidates are selected.

Is your feature request related to a problem? Please describe.

At the moment Windows 11 IME interface and candidates are not recognized.

Describe the solution you'd like

Recognize Windows 11 modern IME interface and items. This requires reistering new UI Automation Id's in overlay class chooser in emoji panel app module and refining element selected event to descend one more level to handle IME candidates in Windows 11.

Describe alternatives you've considered

No IME UI recognition.

Additional context

Fix is included in Windows App Essentials ad-on. Allowing IME candidates to be recognized allows folks to add further refinements.