Today's Plan

• What we've learned

• The future of Language and Computers

• Your future in NLP

What we've learned

We've covered a huge amount of material this quarter

• Machine Learning
• Acoustic and Articulatory Phonetics
• Automatic Speech Recognition
• Text-to-Speech Synthesis
• Unix
• Language Modeling
• Morphosyntax
• POS Tagging and Dependency Parsing
• Lexical Semantics and Word Sense
• Semantic Parsing
• Event Semantics
• Pragmatics
• Conversational Interaction

We started the quarter with an example

"Hey Siri, how long to drive to work?"

• "Traffic to work is light, so it should take 10 minutes via Voigt drive"

This is amazing

... but we now know how it was done!

What did I say? (Acoustic and Articulatory Phonetics)

"Hey Siri, how long to drive to work?"

How did she turn that into text? (Automatic Speech Recognition)

"Hey Siri, how long to drive to work?"

• What parts of it were most difficult?

What parts of speech are there?

"Hey Siri, how long to drive to work?"

How did she figure out what parts of speech were there? (Automatic POS Tagging)

"Hey Siri, how long to drive to work?"

• How/WRB long/RP to/TO drive/VB to/TO work/NN

What kind of syntactic information did she extract? (Dependency parsing)

"Hey Siri, how long to drive to work?"

What word senses did she decide on? (Word Sense Disambiguation)

"Hey Siri, how long to drive to work?"

• drive.v.01 - 'Operate a vehicle'
• workplace.n.01 - 'A place where work is done'

What arguments does that verb expect? (Lexical Semantics)

"Hey Siri, how long to drive to work?"

• Arg0-PAG: driver (vnrole: 11.5-agent)
• Arg1-PPT: vehicle or path (vnrole: 11.5-theme)

How do those arguments map onto the sentence? (Shallow Semantic Parsing)

"Hey Siri, how long to drive to work?"

What queries do we form from this? (Information Retrieval)

• We need ARGM-TMP of 'drive' when ARGM-GOL == 'work'

• ARG0-PAG == $USER

• $USERLOC == getlocation($DEVICE)

• ARGM-GOL == $WORKLOC == 9500 Gilman Drive 92093

• "Get ($TRAFFICSTATUS,$DRIVETIME,$DRIVEROUTE) for navigate($USERLOC,$WORKLOC,car)"

Response Generation (Language Modeling and Pragmatics and Interaction Design)

• Traffic to $ARGM-GOL is $TRAFFICSTATUS, so it should take $DRIVETIME minutes via $DRIVEROUTE

• "Hey Siri, how long to drive to work?" "Traffic to work is light, so it should take 10 minutes via Voigt drive"

Response Playback (Text-to-Speech Synthesis)

"Hey Siri, how long to drive to work?" "Traffic to work is light, so it should take 10 minutes via Voigt drive"

This remains an incredibly complicated process

• ... but you now understand the concepts underlying most of it

... and in the process, you've now covered many of the principal subfields in human linguistics

These systems are not perfect, though

• There is much room for improvement

The Future of Virtual Assistants and NLP

Better NLP functionality

• Improved ASR accuracy

• Improved TTS quality

• Better automatic parsing and POS tagging

• Improvements to Lexical Resources

• Richer semantic parses

Greater breadth of language coverage

• Duplication and expansion of existing resources for other world languages

• Attempts at offering such a service to signed language users

• Improved NLP and Virtual Assistant resources for less wealthy languages

Reduced 'Brittleness' to variability

• Less of a 'happy path' effect

• Improved ability to handle a greater diversity of phrasings of existing commands

• Improved ability to handle a wider diversity of voices and accents

• Improved ability to cope with subadjacency constructions

• Improved conversational repair

Better reach

• Improved ability to interact with a wider number of systems

• Improved integration with existing systems

• More 'skills' to accomplish more tasks

More safeguards

• This tech can be very good, but also very bad

• We need to worry a bit about how to prevent incomplete AI from hurting people

Practical Improvements

• Offline functionality for ASR and TTS and basic question-answering

• Enhanced privacy protections, allowing deeper learning with little trouble

• Improvements to wake-word technology allowing even more fluid access and lower idle power consumption

• Increased ubiquity at decreased cost

More senses and sensors

• Ability to refer to visual data

• "What's that over there?"

• Leveraging existing sensor meshes

Aside: This is how humans work

• We are all vats of brain matter sitting in a dark hole surrounded by bone

• We build our understanding from a mesh of sensor data

• The 'real world' is just inference from input data

Deeper Understanding and Better Inference

• Improved personalized knowledge to enhance inference on real-world topics

• Improved discourse context, to allow this to feel more like a 'relationship'

• Improved understanding of the person's actual life

Greater Agency

• "The next bus will arrive at Gilman and Eucalyptus in 10 minutes. I'll text Jessica and tell her you're on your way."

• "It's time to leave for the airport. I've called a Lyft. Don't forget your passport and glasses, leave your pocketknife at home, and make sure to take your good razor out of your toiletry bag if you're using a carry-on."

• "Hey, it's time to see the Dentist. Dr. Pradhan's got availability next Tuesday at 2pm, and so do you. Should I confirm?"

• "It's Valentine's Day next week. You should remember to grab a card for Jessica. Would you like me to order some dark red roses?"

Better Interaction

• Long term knowledge of preferences and desires

• Better knowledge of the people, places, and things in the user's life

• Improved conversational latency

• More natural-feeling conversation

More dynamic conversation

• Once we have the basics down, we can make these conversations more natural

• Changes to dialect, word choice, etc

• More conversational paths and approaches

• Less 'wooden' interactions

Artificial Intelligence

• General Artificial Intelligence may solve many of these problems

  • ... and will certainly improve every aspect of these

• Even an artificial idiot solves many of them

• AI comes with its own difficulties

... but the simple fact is ...

Computers need to get better at human language

This will require advances in computing

• Improved machine learning

• Improved resource efficiency

• Better Digital Signal Processing

• Better knowledge representations

... and advances in Linguistics

• Better understanding of speech

• Better language modeling

• Better models of semantics and semantic inference

This is your job, now

You'll need to prepare yourself

Useful Linguistics Courses

• LIGN 101 (Intro to Linguistics)

• LIGN 110 (Articulatory Phonetics)

• LIGN 121 (Syntax)

• LIGN 130 (Semantics)

Useful computational follow-up courses

• LIGN 17 (Making and Breaking Codes)

• LIGN 165 (Computational Linguistics)

• LIGN 167 (Deep Learning for Natural Language Understanding)

• LIGN 168 (Computational Speech Processing)

• ... and lots of grad classes, coming soon!

Also consider the Computational Social Sciences Minor or MS

• Focuses on data-driven problems and work throughout the social sciences

• NLP is a part of it, but there's lots more!

• https://css.ucsd.edu/

You have many paths

• Academia and Research

• Government

• Industry

... but these are exciting times

First, we had keyboards

• Then, mice changed everything

• Now, we are on the verge of interacting with our computers in the same way we interact with everything else

Oh my!

Thank you!