skylight-api

Changelog

[0.9]

• Added new field in layers (arp, bootp, citrix_channels, citrix, databases, dce_rpc, dns, email, ftp, http, icmp, ipsec, kerberos, ldap, non_ip, ntlm, other_ip, rdp, smb, socks5, ssh, tcp, tls, udp, vnc, voip):
  • Added new field source.application.
• Changed fields in dce_rpc layer:
  • Changed field dcom.argument to dcom.arguments.
  • Changed field dcom.argument_length to dcom.arguments_lengths.
  • Changed type of dcom.name (./definitions#field-dcom.name).
• Added new fields in bootp layer:
  • Added new field renewal.time.
  • Added new field rebinding.time.
• Added new fields in tcp layer:
  • Added new field client.seqnum_frames.
  • Added new field server.seqnum_frames.
  • Added new field client.initial.rtt.count.
  • Added new field server.initial.rtt.count.
  • Added new field client.initial.rtt.sum.
  • Added new field server.initial.rtt.sum.
  • Added new field client.initial.rtt.square_sum.
  • Added new field server.initial.rtt.square_sum.
  • Added new field client.single.rtt.count.
  • Added new field server.single.rtt.count.
  • Added new field client.single.rtt.sum.
  • Added new field server.single.rtt.sum.
  • Added new field client.single.rtt.square_sum.
  • Added new field server.single.rtt.square_sum.
  • Added new field client.multiple.rtt.count.
  • Added new field server.multiple.rtt.count.
  • Added new field client.multiple.rtt.sum.
  • Added new field server.multiple.rtt.sum.
  • Added new field client.multiple.rtt.square_sum.
  • Added new field server.multiple.rtt.square_sum.
  • Added new field client.retrans.pdus.
  • Added new field server.retrans.pdus.
• Added new model version v38:
  • Changed field ftp.response.code to ftp.reply_codes.
  • Added new field ftp.unknown_query.
  • Added new field ftp.has_password.
  • Changed field ftp.query to ftp.commands.
  • Added new field ftp.commands.counters.
  • Added new field ftp.reply_codes.counters.
• Support of IN operation with Set:
  • Operations as traffic FROM email WHERE foo@bar.com IN recipients are now valid.
• Added new layer email:
  • Added new field email.command.
  • Added new field email.command.counter.
  • Added new field email.response.code.
  • Added new field client.
  • Added new field subject.
  • Added new field date.
  • Added new field sender.username.
  • Added new field sender.address.
  • Added new field recipients.from.
  • Added new field recipients.cc.
  • Added new field recipients.
  • Added new field email.message.id.
  • Added new field attachments.filename.
  • Added new field attachments.content_type.
  • Added new field attachments.sha256.
• Added new layer RDP:
  • Reused field from TLS client.ja3.
  • Reused field from TLS server.ja3.
  • Reused field from TLS alert_types.
  • Added new field rdp.authentications.count.
  • Added new field cookie.
  • Added new field [rdp.known_alerts.count] (./definitions#field-rdp.known_alerts.count).
  • Added new field [rdp.unknown_alerts.count] (./definitions#field-rdp.unknown_alerts.count).
  • Added new field rdp.client.application.count.
  • Added new field rdp.server.application.count.

[0.8]

Added

• Added new layer NTLM:
  • Added new field ntlm.source_proto.
  • Added new field ntlm.negotiate_flags.
  • Added new field netbios.computer_name.
  • Added new field netbios_domain.
  • Added new field dns_computer.
  • Added new field dns_domain.
  • Added new field dns_tree.
  • Added new field target.
  • Added new field server.challenge.
  • Added new field timestamp.
• Added new layer SSH:
  • Added new field ssh.client.version.
  • Added new field ssh.server.version.
  • Added new field ssh.client.hassh.
  • Added new field ssh.server.hassh.
  • Added new field ssh.authentications.count.
  • Added new field ssh.authentications.success.
  • Added new field ssh.key_exchange.agreement.
  • Added new field ssh.min_time_to.newkey.
  • Added new field ssh.max_time_to.newkey.
  • Added new field ssh.min_time_to.login.
  • Added new field ssh.max_time_to.login.
  • Added new field ssh.login.packet_size.
• Added new layer DCE/RPC:
  • Added new field dcerpc.interface.
  • Added new field opnums.
  • Added new field functions.
  • Added new field policy_handle.
  • Added new field server_handle.
  • Added new field dcerpc.computer_name.
  • Added new field challenge.
  • Added new field return_codes.
  • Added new field dcerpc.successes.
  • Added new field dcerpc.errors.
  • Added new field dcom.lcid.
  • Added new field dcom.flags.
  • Added new field dcom.name.
  • Added new field dcom.argument.
  • Added new field dcom.argument_length.
• Added new layer IPSEC:
  • Added new field ipsec.authentication.method.
  • Added new field ipsec.certificate.type.
  • Added new field ipsec.domain.fqdn.
  • Added new field ipsec.encryption.type.
  • Added new field ipsec.esp.
  • Added new field ipsec.group.description.
  • Added new field ipsec.group.type.
  • Added new field ipsec.hash.type.
  • Added new field ipsec.ike.
  • Added new field ipsec.initiator.cookie.
  • Added new field ipsec.key_exchange.
  • Added new field ipsec.key_exchange.count.
  • Added new field ipsec.life.duration.
  • Added new field ipsec.life.type.
  • Added new field ipsec.nat.
  • Added new field ipsec.nonce.
  • Added new field ipsec.responder.cookie.
  • Added new field ipsec.transform.id.
  • Added new field ipsec.transform.type.
  • Added new field ipsec.user.fqdn.
  • Added new field ipsec.vendors.
  • Added new field ipsec.version.
• Added new layer SOCKS5:
  • Added new field socks5.authentication_method.
  • Added new field socks5.command.
  • Added new field socks5.address.type.
  • Added new field socks5.response.status.
  • Added new field target.port.
  • Added new field socks5.target.ip.
  • Added new field has_password.
• Added new layer VNC:
  • Added new field vnc.client.version
  • Added new field vnc.server.version
  • Added new field vnc.source.version
  • Added new field vnc.dest.version
  • Added new field vnc.client.version.major
  • Added new field vnc.server.version.major
  • Added new field vnc.source.version.major
  • Added new field vnc.dest.version.major
  • Added new field vnc.client.version.minor
  • Added new field vnc.server.version.minor
  • Added new field vnc.source.version.minor
  • Added new field vnc.dest.version.minor
  • Added new field vnc.encryption
  • Added new field vnc.supported.encryptions
  • Added new field authentication.result
  • Added new field authentication.status
  • Added new field hostname
• Added new layer FTP:
  • Added new field ftp.command.
  • Added new field ftp.response.code.
  • Added new field ftp.file.sha256.
• Added new layer ARP:
  • Added new field sender.mac.
  • Added new field target.mac.
  • Added new field sender.ip.
  • Added new field target.ip.
  • Added new field sender.hostname.
  • Added new field target.hostname.
• Added new layer LDAP:
  • Added new field message.id.
  • Added new field ldap.type.
  • Added new field authorization.type.
  • Added new field user.
  • Added new field result_code.
  • Added new field filter.
  • Added new field object_name.
  • Added new field search_result_num.
  • Added new field request_attributes.
• Added new DNS fields:
  • Added new field flags.
  • Added new field flag_names.
  • Added new field nb_questions.
  • Added new field nb_answers.
  • Added new field resolved_ip.
  • Added new field ttl.
• Added new HTTP fields:
  • Added new field referrer.
  • Added new field content_disposition.
  • Added new field content_description.
  • Added new field http.version.
  • Added new field http.version.major.
  • Added new field http.version.minor.
• Added new SMB fields:
  • added new field smb.desired_access.
  • added new field smb.file_attributes.
  • Added new field smb.create_options.
  • Added new field smb.creation_time.
  • Added new field smb.last_access_time.
  • Added new field smb.last_write_time.
  • Added new field smb.change_time.
  • Added new field smb.info_type.
  • Added new field smb.file_info_class.
  • Added new field smb.file_info_class.code.
  • Added new field smb.new_file_name.
  • Added new field smb.delete_on_close.
• Added new VoIP fields:
  • added new field callee.hostname.
  • added new field caller.hostname.
• Added new layer Kerberos:
  • added new field client.name.
  • added new field client.realm.
  • added new field etype_ticket.
  • added new field response.etypes.
  • added new field request.etypes.
  • added new field include_pac.
  • added new field kdc_options.
  • added new field kerberos.type.
  • added new field kerberos.msg_types.
  • added new field kerberos.other_count.
  • added new field kerberos.realm.
  • added new field kerberos.response.cipher.
  • added new field kerberos.reply_count.
  • added new field kerberos.request.cipher.
  • added new field server.name.
  • added new field source_proto.
  • added new field renew.time.
  • added new field renew.time.is_illformed.
  • added new field expiration.
  • added new field expiration.is_illformed.
  • added new field server.time.
  • added new field server.time.is_illformed.

Renamed

• Renamed field dcerpc to dcerpc_uuid.encryption

[0.7]

Added

• Support flatten function for url type.
• Added new VLAN related fields. It is possible now to use the operator IN with a list of VLANs, For example, traffic FROM tcp WHERE 5 IN server.vlans.
  • Added new field client.vlans.
  • Added new field server.vlans.
  • Added new field source.vlans.
  • Added new field dest.vlans.
  • Added new field client.vlans.outer.
  • Added new field server.vlans.outer.
  • Added new field source.vlans.outer.
  • Added new field dest.vlans.outer.
  • Added new field client.vlans.inner.
  • Added new field server.vlans.inner.
  • Added new field source.vlans.inner.
  • Added new field dest.vlans.inner.
  • Added new field client.vlans.count.
  • Added new field server.vlans.count.
  • Added new field source.vlans.count.
  • Added new field dest.vlans.count.
• Added new fields for time window exclusion:
  • Added new field time_exclusion.business_hours.
  • Added new field time_exclusion.maintenance_windows.
  • Added new field time_exclusion.any.
• Added new fields for DNS-issued hostnames:
  • Added new field client.hostname.
  • Added new field server.hostname.
  • Added new field source.hostname.
  • Added new field dest.hostname.
  • Added new field netflow.hostname.

Deprecated

• Single VLAN fields are deprecated by fields containing lists of VLANs.
  • Deprecated field client.vlan.
  • Deprecated field server.vlan.
  • Deprecated field source.vlan.
  • Deprecated field dest.vlan.

[0.6]

Added

• Added new endpoint query-cancel
• Support flatten function for zone_id type.
• Added new time related fields:
  • Added new field begin.
  • Added new field end.
  • Added new field request.begin.
  • Added new field request.end.
  • Added new field query.begin.
  • Added new field query.end.
  • Added new field page.begin.
  • Added new field page.end.

Removed

• points function has been removed, field points should be used instead.

[0.5.1]

Added

• Added new field smb.md5
• Added new endpoint get-degradations
• Added new endpoint get-layers

[0.5]

Added

• Added new zone related fields:
  • Added new field client.zone.id.
  • Added new field server.zone.id.
  • Added new field source.zone.id.
  • Added new field dest.zone.id.
  • Added new field client.error.zone.id.
  • Added new field server.error.zone.id.
  • Added new field source.error.zone.id.
  • Added new field dest.error.zone.id.
  • Added new field caller.zone.id.
  • Added new field callee.zone.id.
• Added new field application.id.
• Add fields related to MD5 for HTTP
  • Add new field request.payload.md5
  • Add new field response.payload.md5

Changed

• Add support for extended IP and MAC masks (<ip>/<ip>, <mac>/<mac>).
• Fix using #(count) operator on complex fields.
• Zone related changes:
  • Renamed field client.zone to client.zone.name.
  • Renamed field server.zone to server.zone.name.
  • Renamed field source.zone to source.zone.name.
  • Renamed field dest.zone to dest.zone.name.
  • Renamed field client.error.zone to client.error.zone.name.
  • Renamed field server.error.zone to server.error.zone.name.
  • Renamed field source.error.zone to source.error.zone.name.
  • Renamed field dest.error.zone to dest.error.zone.name.
  • Renamed field caller.zone to caller.zone.name.
  • Renamed field callee.zone to callee.zone.name.
• Renamed field application to application.name.

[0.4] - 2020-05-27

Added

• Added new field capture.hostname.
• Added new field caller.label.
• Added new field callee.label.
• Added new field client.ja3.
• Added new field server.ja3.
• Added new field source.ja3.
• Added new field dest.ja3.

Changed

• Renamed field capture to capture.id.
• Clause FROM is now mandatory.
• Improve support for mac address querying:
  • Able to match a mac address using both a continuous and non continuous mask.
  • Create non continuous mask filtering for mac address.
  • Implement IN operation for mac address using a continuous mask.
• Order results in PVQL distinct sets.
• Add PVQL setting limit_size_set = 100.
• Implement glob/iglob functions for applications.
• Ignoring case when sorting by a string field.
• Fix can’t query dicts as values.

Removed

• Removed field storage from public API.