Referring to "type" as both a meta-type (built-in, enum, union,
alternate, or struct) and a specific type (the name that the
schema uses for declaring structs) is confusing. Finish up the
conversion to using "struct" in qapi schema by removing the hack
in the generator that allowed 'type'.
Backports commit 3e391d355644b2bff7c9f187759aadb46c6e051f from qemu
Python 2 and Python 3 have a wild history of whether strings
default to ascii or unicode, where Python 3 requires checking
isinstance(foo, basestr) to cover all strings, but where that
code is not portable to Python 2. It's simpler to just state
that we don't care about Unicode strings, and to just always
use the simpler isinstance(foo, str) everywhere.
I'm no python expert, so I'm basing it on this conversation:
https://lists.gnu.org/archive/html/qemu-devel/2014-09/msg05278.html
Backports commit fe2a9303c9e511462f662a415c2e9d2defe9b7ca from qemu
Referring to "type" as both a meta-type (built-in, enum, union,
alternate, or struct) and a specific type (the name that the
schema uses for declaring structs) is confusing. The confusion
is only made worse by the fact that the generator mostly already
refers to struct even when dealing with expr['type']. This
commit changes the generator to consistently refer to it as
struct everywhere, plus a single back-compat tweak that allows
accepting the existing .json files as-is, so that the meat of
this change is separate from the mindless churn of that change.
Fix the testsuite fallout for error messages that change, and
in some cases, become more legible. Improve comments to better
match our intentions where a struct (rather than any complex
type) is required. Note that in some cases, an error message
now refers to 'struct' while the schema still refers to 'type';
that will be cleaned up in the later commit to the schema.
Backports commit fd41dd4eae5f7ea92f10c04cb3f217727fcee91f from qemu
Now that we have a way to validate every type, we can also be
stricter about enforcing that callers that want to bypass
type safety in generated code. Prior to this patch, it didn't
matter what value was associated with the key 'gen', but it
looked odd that 'gen':'yes' could result in bypassing the
generated code. These changes also enforce the changes made
earlier in the series for documentation and consolidation of
using '**' as the wildcard type, as well as 'gen':false as the
canonical spelling for requesting type bypass.
Note that 'gen':false is a one-way switch away from the default;
we do not support 'gen':true (similar for 'success-response').
In practice, this doesn't matter.
Backports commit 2cbf09925ad45401673a79ab77f67de2f04a826c from qemu
...or an array of dictionaries. Although we have to cater to
existing commands, returning a non-dictionary means the command
is not extensible (no new name/value pairs can be added if more
information must be returned in parallel). By making the
whitelist explicit, any new command that falls foul of this
practice will have to be self-documenting, which will encourage
developers to either justify the action or rework the design to
use a dictionary after all.
It's a little bit sloppy that we share a single whitelist among
three clients (it's too permissive for each). If this is a
problem, a future patch could tighten things by having the
generator take the whitelist as an argument (as in
scripts/qapi-commands.py --legacy-returns=...), or by having
the generator output C code that requires explicit use of the
whitelist (as in:
then having the callers define appropriate macros). But until
we need such fine-grained separation (if ever), this patch does
the job just fine.
Backports commit 10d4d997f86cf2a4ce89145df5658952d5722e56 from qemu
Previous commits demonstrated that the generator overlooked various
bad naming situations:
- types, commands, and events need a valid name
- enum members must be valid names, when combined with prefix
- union and alternate branches cannot be marked optional
Valid upstream names match [a-zA-Z][a-zA-Z0-9_-]*; valid downstream
names match __[a-zA-Z][a-zA-Z0-9._-]*. Enumerations match the
weaker [a-zA-Z0-9._-]+ (in part thanks to QKeyCode picking an enum
that starts with a digit, which we can't change now due to
backwards compatibility). Rather than call out three separate
regex, this patch just uses a broader combination that allows both
upstream and downstream names, as well as a small hack that
realizes that any enum name is merely a suffix to an already valid
name prefix (that is, any enum name is valid if prepending _ fits
the normal rules).
We could reject new enumeration names beginning with a digit by
whitelisting existing exceptions. We could also be stricter
about the distinction between upstream names (no leading
underscore, no use of dot) and downstream (mandatory leading
double underscore), but it is probably not worth the bother.
Backports commit c9e0a798691d8c45747b082206e789c8f50523c9 from qemu
Now that we know every expression is valid with regards to
its keys, we can add further tests that those keys refer to
valid types. With this patch, all uses of a type (the 'data':
of command, type, union, alternate, and event; the 'returns':
of command; the 'base': of type and union) must resolve to an
appropriate subset of metatypes declared by the current qapi
parse; this includes recursing into each member of a data
dictionary. Dealing with '**' and nested anonymous structs
will be done in later patches.
Backports commit dd883c6f0547f02ae805d02852ff3691f6d08f85 from qemu
In the near term, we will use it for a sensible-looking
'gen':false inside command declarations, instead of the
current ugly 'gen':'no'.
In the long term, it will allow conversion from shorthand
with defaults mentioned only in side-band documentation:
'data':{'*flag':'bool', '*string':'str'}
into an explicit default value documentation, as in:
'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
'string':{'type':'str', 'optional':true, 'default':null}}
We still don't parse integer values (also necessary before
we can allow explicit defaults), but that can come in a later
series.
Backports commit e53188ada516c814a729551be2448684d6d8ce08 from qemu
The previous commit demonstrated that the generator overlooked
duplicate expressions:
- a complex type or command reusing a built-in type name
- redeclaration of a type name, whether by the same or different
metatype
- redeclaration of a command or event
- collision of a type with implicit 'Kind' enum for a union
- collision with an implicit MAX enum constant
Since the c_type() function in the generator treats all names
as being in the same namespace, this patch adds a global array
to track all known names and their source, to prevent collisions
before it can cause further problems. While valid .json files
won't trigger any of these cases, we might as well be nicer to
developers that make a typo while trying to add new QAPI code.
Backports commit 4dc2e6906e1084fdd37bf67385c5dcd2c72ae22b from qemu
The previous commit demonstrated that the generator overlooked some
fairly basic broken expressions:
- missing metataype
- metatype key has a non-string value
- unknown key in relation to the metatype
- conflicting metatype (this patch treats the second metatype as an
unknown key of the first key visited, which is not necessarily the
first key the user typed)
Add check_keys to cover these situations, and update testcases to
match. A couple other tests (enum-missing-data, indented-expr) had
to change since the validation added here occurs so early.
Conversely, changes to ident-with-escape results show that we still
have problems where our handling of escape sequences differs from
true JSON, which will matter down the road if we allow arbitrary
default string values for optional parameters (but for now is not
too bad, as we currently can avoid unicode escaping as we don't
need to represent anything beyond C identifier material).
While valid .json files won't trigger any of these cases, we might
as well be nicer to developers that make a typo while trying to add
new QAPI code.
Backports commit 0545f6b8874c28d97369f2c83e5077e0461d4f12 from qemu
Previous patches have led up to the point where I create the
new meta-type "'alternate':'Foo'". See the previous patches
for documentation; I intentionally split as much work into
earlier patches to minimize the size of this patch, but a lot
of it is churn due to testsuite fallout after updating to the
new type.
Backports commit ab916faddd16f0165e9cc2551f90699be8efde53 from qemu
Special-casing 'discriminator == {}' for handling anonymous unions
is getting awkward; since this particular type is not always a
dictionary on the wire, it is easier to treat it as a completely
different class of type, "alternate", so that if a type is listed
in the union_types array, we know it is not an anonymous union.
This patch just further segregates union handling, to make sure that
anonymous unions are not stored in union_types, and splitting up
check_union() into separate functions. A future patch will change
the qapi grammar, and having the segregation already in place will
make it easier to deal with the distinct meta-type.
Backports commit 811d04fd0cff1229480d3f5b2e349f646ab6e3c1 from qemu
This patch widens the scope of a try block (with the attending
reindentation required by Python) in preparation for a future
patch adding more instances of QAPIExprError inside the block.
It's easier to separate indentation from semantic changes, so
this patch has no real behavior change.
Backports commit 268a1c5eb10832c2e4476d3fe199ea547dabecb7 from qemu
Previous commits demonstrated that the generator had several
flaws with less-than-perfect unions:
- a simple union that listed the same branch twice (or two variant
names that map to the same C enumerator, including the implicit
MAX sentinel) ended up generating invalid C code
- an anonymous union that listed two branches with the same qtype
ended up generating invalid C code
- the generator crashed on anonymous union attempts to use an
array type
- the generator was silently ignoring a base type for anonymous
unions
- the generator allowed unknown types or nested anonymous unions
as a branch in an anonymous union
Backports commit 44bd1276a7dea747c41f250cb71ab65965343a7f from qemu
None of the existing QMP or QGA interfaces uses a union with a
base type but no discriminator; it is easier to avoid this in the
generator to save room for other future extensions more likely to
be useful. An earlier commit added a union-base-no-discriminator
test to ensure that we eventually give a decent error message;
likewise, removing UserDefUnion outright is okay, because we moved
all the tests we wish to keep into the tests of the simple union
UserDefNativeListUnion in the previous commit. Now is the time to
actually forbid simple union with base, and remove the last
vestiges from the testsuite.
Backports commit a8d4a2e4d7e1a0207699de47142c9bdbf2cc8675 from qemu
The previous commit demonstrated that the generator had several
flaws with less-than-perfect enums:
- an enum that listed the same string twice (or two variant
strings that map to the same C enumerator) ended up generating
an invalid C enum
- because the generator adds a _MAX terminator to each enum,
the use of an enum member 'max' can also cause this clash
- if an enum omits 'data', the generator left a python stack
trace rather than a graceful message
- an enum that used a non-array 'data' was silently accepted by
the parser
- an enum that used non-string members in the 'data' member
was silently accepted by the parser
Add check_enum to cover these situations, and update testcases
to match. While valid .json files won't trigger any of these
cases, we might as well be nicer to developers that make a typo
while trying to add new QAPI code.
Backports commit cf3935907b5df16f667d54ad6761c7e937dcf425 from qemu
We were missing the 'size' builtin type (which means that QAPI using
[ 'size' ] would fail to compile).
Backports commit cb17f79eef0d161e81ac457e4c1f124405be2a18 from qemu
There was some redundancy between builtin_types[] and
builtin_type_qtypes{}. Merge them into one.
Backports commit b52c4b9cf0bbafdf8cede4ea1f62770d86815718 from qemu
Defaulting a parameter to True, then having all callers omit or
pass an explicit True for that parameter, is pointless. Looks
like it has been dead since introduction in commit 06d64c6, more
than 4 years ago.
Backports commit 6540e9f35bfeea2baf4509745516172070dca412 from qemu
No backend was setting an error when ending the visit of a list or
implicit struct, or when moving to the next list node. Make the
callers a bit easier to follow by making this a part of the contract,
and removing the errp argument - callers can then unconditionally end
an object as part of cleanup without having to think about whether a
second error is dominated by a first, because there is no second
error.
A later patch will then tackle the larger task of splitting
visit_end_struct(), which can indeed set an error.
Backports commit 08f9541dec51700abef0c37994213164ca4e4fc9 from qemu
