Upgraded Technical Features of Python

Introduction:

Python is one of the most popular programming languages used by developers worldwide. Its simplicity, flexibility, and ease of use make it a preferred choice for both beginners and experienced developers. Python has been evolving continuously, with new versions and upgrades coming up regularly. The language is widely used in various fields, including scientific computing, data analysis, web development, machine learning, and more.

On October 24, 2022, Python 3.11 was released. The most recent version of Python is quicker and easier to use. Its preparation for use in the real world took seventeen months.

The latest version of Python, 3.11, has a tonne of updates and enhancements. Explore the coolest and most useful new features right here.

• More descriptive tracebacks and improved error messages
• Lots of effort in the Faster CPython project result in faster code execution.
• Task and exception groups that make using asynchronous code easier
• There are several new typing features that make static typing support in Python better.
• Native TOML, or Tom's Obvious Minimal Language, support for working with configuration files.

More Descriptive Tracebacks:

With its accessible syntax and robust data structures, Python is frequently regarded as a suitable newbie programming language. How to read the traceback that is shown when Python encounters an error can be difficult for anyone, but it can be especially difficult for people who are new to Python.

Python's error messages were considerably enhanced in version 3.10. The most anticipated feature of Python 3.11 will likewise improve the developer experience, in a similar manner. The tracebacks have decorative annotations that can speed up error message interpretation.

Add the following code to a file called inverse.py to see a brief demonstration of the improved traceback:

# inverse.py

def inverse(number):

    return 1 / number

print(inverse(0))

To determine a number's multiplicative inverse, use the inverse() function. The multiplicative inverse of 0 doesn't exist, so when you run your code, an error is raised:

$ python inverse.py

Traceback (most recent call last):

  File "/home/realpython/inverse.py", line 6, in <module>

    print(inverse(0))

          ^^^^^^^^^^

  File "/home/realpython/inverse.py", line 4, in inverse

    return 1 / number

           ~~^~~~~~~~

ZeroDivisionError: division by zero

The traceback contains the symbols ^ and ~ packed within it. They are employed to direct your focus to the code that is the source of the mistake. You should always begin at the bottom and work your way up when performing a traceback. In this case, the division 1 / number results in a ZeroDivisionError. Since 0 has no inverse, invoking inverse(0) is the real offender.

Having this extra assistance in identifying errors is beneficial. If your code is more complicated, the annotated tracebacks are much more effective. They could be able to provide details that the traceback alone couldn't before.

Faster Code Execution

Python is known for being a sluggish language. A normal loop, for instance, runs orders of magnitude slower in Python than it would in C. There are numerous strategies to overcome this disadvantage. Code execution speed often becomes less significant than programmer productivity.

Additionally, Python is excellent at encasing libraries created in faster languages. For instance, NumPy calculations are substantially faster than equivalent pure Python calculations. This makes Python a formidable competitor in the data science arena when combined with how simple it is to write code.

Nevertheless, there has been a drive to speed up the Python core language. Mark Shannon made many performance suggestions that Python developers could implement in the fall of 2020. The Shannon Plan, which aims to make Python five times quicker across several versions, is an extremely ambitious plan.

A group of programmers working on the Faster CPython project, as it is now known, including Mark Shannon and Python's founder, Guido van Rossum, have Microsoft's support. Python 3.11, which is based on the Faster CPython project, has a lot of enhancements.

A specialized adaptable interpreter is described in PEP 659. The basic goal is to accelerate running code by optimizing frequently performed tasks. With the exception of not affecting compilation, this is comparable to just-in-time (JIT) compilation. Rather, Python's bytecode is modified or altered as needed.

Before being executed, Python code is converted to bytecode. Each line of Python code is split up into many bytecode statements because bytecode contains more fundamental instructions than conventional Python code.

You can use dis to view the bytecode for Python. Consider a function that can convert between feet and meters

def feet_to_meters(feet):

 2    return 0.3048 * feet

 3

You can disassemble this function into bytecode by calling dis.dis():

>>>

>>> import dis

>>> dis.dis(feet_to_meters)

  1           0 RESUME                   0

  2           2 LOAD_CONST               1 (0.3048)

              4 LOAD_FAST                0 (feet)

              6 BINARY_OP                5 (*)

             10 RETURN_VALUE

One bytecode instruction is detailed on each line. The line number, byte address, operation code name, operation parameters, and a paraphrase of the parameters are the five columns.

To write Python, you typically don't need to be familiar with bytecode. However, it can assist you in comprehending the inner workings of Python.

The bytecode creation now includes a new phase called speeding. This substitutes adaptive instructions for instructions that could be runtime optimized. Each of these instructions will consider its application before specializing.

Once a function has been invoked a predetermined number of times, the quickening begins. This occurs in CPython 3.11 after eight invocations. By using the adaptive argument while calling dis(), you can see how the interpreter modifies bytecode. Create a function first, then seven times call it with arguments that are floating-point numbers:

>>> def feet_to_meters(feet):

...    return 0.3048 * feet

...

>>> feet_to_meters(1.1)

0.33528

>>> feet_to_meters(2.2)

0.67056

>>> feet_to_meters(3.3)

1.00584

>>> feet_to_meters(4.4)

1.34112

>>> feet_to_meters(5.5)

1.6764000000000001

>>> feet_to_meters(6.6)

2.01168

>>> feet_to_meters(7.7)

2.34696

Next, have a look at the bytecode of feet_to_meters():

>>>

>>> import dis

>>> dis.dis(feet_to_meters, adaptive=True)

  1           0 RESUME                   0

  2           2 LOAD_CONST               1 (0.3048)

              4 LOAD_FAST                0 (feet)

              6 BINARY_OP                5 (*)

             10 RETURN_VALUE

You won’t observe anything special yet. This version of the bytecode is still the same as the non-adaptive one. That changes when you call feet_to_meters() the eighth time:

>>>

>>> feet_to_meters(8.8)

2.68224

>>> dis.dis(feet_to_meters, adaptive=True)

  1           0 RESUME_QUICK                 0

  2           2 LOAD_CONST__LOAD_FAST        1 (0.3048)

              4 LOAD_FAST                    0 (feet)

              6 BINARY_OP_MULTIPLY_FLOAT     5 (*)

             10 RETURN_VALUE

Many of the original instructions have now been changed with more specific ones. To multiply two float integers more quickly, BINARY_OP has been specialized to BINARY_OP_MULTIPLY_FLOAT.

Despite the fact that feet_to_meters() has been modified to be more efficient when feet is a float parameter, it continues to function normally for other parameter types by reverting to the previous bytecode instruction. Although the internal processes have changed, your code will continue to operate precisely as it did before.

The specific instructions continue to adapt. fifty-two additional calls to your function, but this time with an integer argument:

>>> for feet in range(52):

...    feet_to_meters(feet)

...

>>> dis.dis(feet_to_meters, adaptive=True)

  1           0 RESUME_QUICK                 0

  2           2 LOAD_CONST__LOAD_FAST        1 (0.3048)

              4 LOAD_FAST                    0 (feet)

              6 BINARY_OP_MULTIPLY_FLOAT     5 (*)

             10 RETURN_VALUE

The two float numbers can still be multiplied by the Python interpreter. One more call to feet_to_meters() with an integer causes it to quit and go back to a general, adaptable instruction:

>>> feet_to_meters(52)

15.8496

>>> dis.dis(feet_to_meters, adaptive=True)

  1           0 RESUME_QUICK              0

  2           2 LOAD_CONST__LOAD_FAST     1 (0.3048)

              4 LOAD_FAST                 0 (feet)

              6 BINARY_OP_ADAPTIVE        5 (*)

             10 RETURN_VALUE

Because one of the operators, 0.3048, is always a floating-point number, the bytecode instruction is altered to BINARY_OP_ADAPTIVE rather than BINARY_OP_MULTIPLY_INT in this instance.

It is more difficult to optimize multiplications between numbers of the same type than it is to multiply integers and floating point numbers. There is currently no specialized instruction for multiplying floats and integers.

You should gain some understanding of how the adaptive specializing interpreter functions from this example. In general, you shouldn't be concerned about altering your current code to benefit from it. The majority of your code will simply execute faster as is.

Having said that, there are a few situations in which you might be able to restructure your code to enable more effective specialization. The Brandt Bucher specialist is a tool that illustrates how the interpreter handles your code. A manual code improvement example is provided in the tutorial. Listen to the Talk Python to Me podcast for further information.

The following principles are crucial for the Faster CPython project:

• There won't be any Python-breaking changes brought about by the project.
• The majority of code should perform better.

According to tests, "CPython 3.11 is on average 25% faster than CPython 3.10" (Source). However, rather than how well Python 3.11 performs on benchmarks, you should be more concerned with how it performs on your own code.

Nicer Syntax for Asynchronous Tasks

The code is still performed one step at a time with asynchronous programming, but the system won't wait for this step to finish before going on to the next.

The Python "asyncio" package takes care of this. As each asynchronous job completes, we wait for it to be collected using asyncio.gather(). Let's run some errands, for instance.

Running it in terminal gives us this output,

However, manually keeping track of a list of jobs before asyncio.gather awaits them is time-consuming. 3.11 adds a new feature/class called TaskGroup().

The TaskGroup serves as a context manager, keeping track of the set of tasks that will be left waiting for them all after they go. Additionally, the syntax is simpler.

PEP 673: The Self Type

Previously, for type hints, we had to specify a type variable manually if we wanted to refer to the current class itself, as illustrated below.

A Self type, which designates the encapsulating class itself, is now available in version 3.11. The trouble of defining a type variable will be removed as a result.

PEP 654: Exception Group

The Exception Group function included in 3.11 provides a comparable exception handling "grouping" capabilities. It can be compared to numerous typical exceptions that have been combined into one.

As we can see, the ExceptionGroup raised both of the sub-exceptions, each of which was presented in its own panel, in response to an error.

Additionally, Python 3.11 introduces a new keyword except* to manage the ExceptionGroup.

Multiple errors contained in the ExceptionGroup are handled by except*. Use of this functionality in "asyncio," when numerous async jobs are executing concurrently, might have a greater impact.

PEP 678: Customized Exception Notes

The exception notes with add_note, another fantastic new tool for error handling, lets you add personalized messages. For instance,

Conclusion

The developer experience gains with Python 3.11—better error messages and quicker code execution—are the main successes. These are excellent incentives to quickly improve the environment you use for local growth. This type of upgrading also entails the least amount of risk because any faults you encounter should only have minor consequences.

Another important reason to update your production environment is the speed gain.

Scope @ N9 IT Solutions:

1. N9 IT Solutions is a leading IT development and consulting firm providing a broad array of customized solutions to clients throughout the United States. 
2. It got established primarily with an aim to provide consulting and IT services in today’s dynamic environment.
3. N9 IT also offers consulting services in many emerging areas like Java/J2ee, Cloud Computing, Database Solutions, DevOps, ERP, Mobility, Big Data, Application Development, Infrastructure Managed Services, Quality Assurance and Testing.